An image quality enhancement method based on adjustable parameters

Abstract

This invention belongs to the field of image processing and proposes an image quality enhancement method based on adjustable parameters. The invention first converts an input low-light, low-contrast RGB image into an HSV image, then uses an image enhancement algorithm to obtain three enhanced images to be fused. Next, image pyramid decomposition and reconstruction are used to fuse the three images. Finally, the fused HSV image is converted back to an RGB image, resulting in a high-contrast, high-brightness image. Because the fusion algorithm is implemented using a hardware pipeline approach, it offers real-time performance and adjustability, and is expected to be applied in fields such as security monitoring and night vision imaging.

Description

Technical Field

[0001] This invention belongs to the field of image processing and relates to an image quality enhancement method based on adjustable parameters. Background Technology

[0002] In actual image shooting, due to the limited dynamic range of digital cameras, it is common for images to be locally or overall dark. For example, in poor lighting conditions at night, images will exhibit low overall brightness and contrast. Or, when shooting in backlight, due to limitations in exposure settings, it is difficult to capture a bright background and a dark foreground target simultaneously.

[0003] With societal development, the demand for high-quality images is increasing daily, particularly in fields such as security monitoring and nighttime imaging. Currently, image fusion technology primarily involves fusing images of the same scene at different exposures or focuses by adjusting aspects such as hue, brightness, and color saturation. This is achieved by assigning weights to different images, normalizing these weights, and then performing a weighted sum. However, this method performs poorly in areas with drastic pixel changes, often producing artifacts. Furthermore, it requires capturing images at different exposures or focuses, making it complex to implement and unable to meet the imaging requirements of security monitoring and nighttime imaging applications.

[0004] Moreover, current image fusion mainly runs on the software side, and the image fusion speed cannot meet the requirements of real-time.

[0005] Therefore, it is necessary to propose an image quality enhancement method based on adjustable parameters to address the shortcomings of existing technologies. Summary of the Invention

[0006] This invention designs and proposes an image quality enhancement method based on adjustable parameters. This method can receive a low-light, low-contrast image, process it through hardware circuitry, and obtain a high-contrast, high-brightness, high-quality image in real time.

[0007] To achieve the above objectives, the present invention provides an image quality enhancement method based on adjustable parameters, comprising:

[0008] First, the pixel stream of an RGB format image to be enhanced is input into the color space conversion module to convert the RGB pixel stream into an HSV pixel stream.

[0009] Next, the HSV image pixel stream is input into the fused image generation module, which will generate the original image, brightness enhancement image, contrast enhancement image, and weights corresponding to the center pixels of their respective m*m pixel matrices. The brightness enhancement coefficient, contrast enhancement coefficient, pixel matrix order m, and Gaussian Laplacian decomposition layer number n can be input from the outside.

[0010] Next, the weights of the original image, the brightness enhancement image, and the contrast enhancement image are input into the n-layer Gaussian image pyramid generation module to obtain an n-layer Gaussian weight pyramid. The original image, the brightness enhancement image, and the contrast enhancement image are then input into the n-layer Laplacian image pyramid generation module to obtain an n-layer Laplacian image pyramid.

[0011] Next, the n-layer Gaussian weight pyramids of the three images are fused with the n-layer Laplacian image pyramids to obtain the enhanced HSV pixel stream. In the fusion process, a layered and block-based processing method is adopted to reduce resource consumption and improve computational efficiency. Specifically, the pixel values ​​at the center of the m*m pixel matrix are used to replace the surrounding pixel values.

[0012] Finally, the HSV pixel stream is input into the color space conversion module to convert the HSV pixel stream into an RGB pixel stream.

[0013] The beneficial effects of this invention are as follows: This invention designs and proposes an image quality enhancement method based on adjustable parameters. Compared with other image fusion methods, it has real-time performance and versatility, and can complete the enhancement and fusion processing of real-time video streams. It is applicable to the fusion enhancement of most low-light and low-contrast scenes, and can effectively enhance the dark details of the image while ensuring image quality. Furthermore, by adjusting the parameters m and n, this method can be adapted to different hardware environment requirements. Attached Figure Description

[0014] The following figures are provided to further illustrate the present invention.

[0015] Figure 1 This is a hardware architecture diagram of an image quality enhancement method based on adjustable parameters.

[0016] Figure 2 This is a detailed architecture diagram of the image fusion generation module hdr_derived_input.

[0017] Figure 3 This is a schematic diagram illustrating the decomposition and reconstruction of a 1920*1080 image using two layers of the Gaussian Pyramid and the Laplace Pyramid.

[0018] Figure 4 These are four cases of the improved mean filtering in this invention.

[0019] Figure 5 The image to be processed.

[0020] Figure 6 This is the processed image. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with the structural schematic diagram of this invention.

[0022] The hardware architecture of an image quality enhancement method based on adjustable parameters according to the present invention is as follows: Figure 1 As shown.

[0023] like Figure 1 The leftmost input shown is an RGB color image under low-light conditions. The color space conversion module `rgb_hsv` separates the luminance, saturation, and hue. The separated luminance information is then processed by the image fusion generation module `hdr_derived_input` to obtain two additional grayscale images, representing the luminance-enhanced and contrast-enhanced images respectively, which are output synchronously with the original luminance image. Furthermore, the `hdr_derived_input` module calculates the weight information for each of the three grayscale images. The weight information w0, w1, w2 and the luminance information g0, g1, g2 output by the module are used for pyramid decomposition according to the input Gaussian image pyramid generation module and Laplacian image pyramid generation module shown in the figure. The resulting n-layer Gaussian weight pyramid and n-layer Laplacian luminance pyramid are input to the image fusion module `fusion`. Each layer is fused and added to the corresponding layers of the other two images to obtain sum1, sum2, sum3, ..., sumn. Figure 1 Taking n=3 as an example, sum1, sum2, sum3, ..., sumn are input into the image reconstruction module reconstruct to reconstruct and obtain the fused luminance image V. At the same time, the hue H and saturation S obtained after the color space conversion in the first step are input into the delay module delay for synchronization delay. The delayed H, S and luminance image V are then input into the color space conversion module hsv_rgb to obtain the fused RGB color image.

[0024] The following is about Figure 1 Each sub-module will be explained in detail.

[0025] The color space conversion module is rgb_hsv. Let (r, g, b) be the three color components: red, green, and blue, each with a value range of [0, 1]. Let max be the maximum value among r, g, and b, and min be the minimum value. To obtain the (h, s, v) values ​​in the HSV color space, where the hue (h) ranges from [0, 360), and the saturation (s) and brightness (l) range from [0, 1], the calculation formula is:

[0026]

[0027]

[0028] The hardware architecture of the image fusion generation module hdr_derived_input is as follows: Figure 2 As shown.

[0029] The image fusion generation module is used to obtain brightness-enhanced and contrast-enhanced images from the grayscale image. Together with the original grayscale image, it outputs a total of three brightness images for subsequent fusion. It also outputs the weight images corresponding to the three grayscale images.

[0030] Figure 2 The contrast enhancement algorithm uses a linear contrast enhancement algorithm to improve image contrast, while the brightness enhancement algorithm uses a gamma brightness enhancement algorithm. Both contrast and brightness can be configured by passing parameters through registers. The contrast of the output image is adjusted accordingly based on the contrast difference of the input image. If the contrast of the input image is poor, the contrast of the output image can be increased by changing the parameters. If the contrast of the input image is good, the contrast of the output image is adjusted slightly or not at all. The brightness enhancement algorithm works similarly.

[0031] Next, the luminance weight and tonal contrast weight are calculated for each of the three grayscale images. Well-exposed pixels have higher weights, while overexposed or underexposed pixels have lower luminance weights. The calculation formulas here primarily consider that most images in the real world follow a Gaussian distribution; simply put, pixels with a value of 128 generally have a higher luminance weight. The contrast weight takes into account not only luminance but also color information, therefore it needs to be calculated using all three HSV channels simultaneously. The formulas for calculating luminance and tonal contrast weights are as follows:

[0032]

[0033] W c (x,y)=I(x,y)(1+cos(αH(x,y)+φ)S(x,y))

[0034] The weight calculation in this invention uses a lookup table, where W b (x,y) represents the brightness weight of pixel I(x,y), W c (x,y) represents the hue contrast weight of pixel I(x,y), H(x,y) represents the hue of pixel I(x,y), S(x,y) represents the saturation of pixel I(x,y), and αφ is an adjustable parameter.

[0035] In the hdr_weight_normalize module, the brightness weight and contrast weight of each image are multiplied together. Then, the weights after multiplying the three images are normalized and output separately, as shown in the formula below.

[0036] W(x,y)=Wb (x,y)W c (x,y)

[0037]

[0038] in W is the normalized weight output for the k-th image. k (x,y) represents the weight of the k-th image.

[0039] The Gaussian and Laplacian pyramid decomposition and reconstruction modules, hdr_gaussian_pyramid_nlevel, hdr_laplacian_pyramid_nlevel, and fusion, are used here as an example to decompose and reconstruct a 1920*1080 image using a two-layer Gaussian pyramid and a Laplacian pyramid. Figure 3 As shown.

[0040] Figure 3 The left side of the image is a weight map (1920x1080) and a gray map (1920x1080). Gaussian decomposition of the weight map yields a second Gaussian pyramid (gaussian 2960x540). The original image is delayed by two rows of pixels to obtain a first Gaussian pyramid (gaussian 11920x1080). Laplacian decomposition of the luminance map yields a second Laplacian pyramid (laplacian 2960x540). The second Laplacian pyramid is upsampled, filtered, and then subtracted from the original image to obtain the first Laplacian pyramid.

[0041] Then, the first and second Gaussian pyramids are multiplied by the Laplacian pyramid to obtain sum2960x540 and sum11920x1080 respectively. sum2960x540 is upsampled and filtered to obtain up_sum21920x1080, and finally summed with sum11920x1080 to obtain the fused and reconstructed brightness map. A block-based approach is used in the multiplication of the Gaussian and Laplacian pyramids. The size m of the block matrix is ​​configured through a register. If the image resolution is large, m can be set larger, and operand isolation techniques are used with an enable signal to save computational resources and reduce power consumption. If the image resolution is small, m can be set to 1, i.e., no block division is performed.

[0042] Furthermore, this is only a decomposition and reconstruction of a brightness image. At the multiply position, two additional images need to be added simultaneously to obtain the complete image. The algorithms used in each module are described below:

[0043] Downsampling module: This design uses an interlaced pixel extraction method, retaining only pixels in odd-numbered rows and columns. An 8-bit wide, 512-bit (should be 0.5 times the current layer image row resolution) asynchronous FIFO is used as a buffer, and the write clock frequency should be 4 times the read clock frequency.

[0044] Upsampling module: The upsampling module design uses a method of padding with zeros every other row and column. It also uses an 8-bit wide, 512-bit (should be 0.5 times the current layer image row resolution) asynchronous FIFO as a buffer. The read clock frequency is 4 times the write clock frequency.

[0045] The improved mean filtering proposed in this invention: After verification, it was found that downsampling without filtering has minimal impact on image decomposition and reconstruction. Therefore, the filtering operation before downsampling can be optimized, and filtering is only performed after upsampling. Since upsampling uses alternating row and column zero padding, directly using ordinary mean filtering would reduce image brightness. Therefore, it has been improved by selecting only valid data for mean calculation. To save resources, a 3*3 sliding window is still used, resulting in four possible scenarios as follows: Figure 4 As shown, the filtering operation can be completed by handling these four cases in a special way.

[0046] The color space conversion module hsv_rgb uses the following formula to convert the color space from HSV to RGB:

[0047]

[0048]

[0049] For each color vector (r, g, b).

[0050]

[0051] Figure 5 , Figure 6 The image shown is a comparison of the effects after processing with a parameter-adjustable image quality enhancement method. It can be clearly seen that after processing, the brightness, contrast, and color of the image are significantly improved while maintaining quality.

[0052] The above description merely illustrates the embodiments of the present invention, but should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. An image quality enhancement method based on adjustable parameters, characterized in that: First, the pixel stream of an RGB format image to be enhanced is input into the color space conversion module to convert the RGB pixel stream into an HSV pixel stream; Next, the HSV image pixel stream is input into the fused image generation module. This module generates the original image, brightness enhancement image, contrast enhancement image, and the weights corresponding to the center pixels of their respective m*m pixel matrices. The brightness enhancement coefficient, contrast enhancement coefficient, pixel matrix order m, and Gaussian Laplacian decomposition layer n are input from the outside. Then, the weights of the original image, brightness enhancement image, and contrast enhancement image are input into the n-layer Gaussian image pyramid generation module to obtain an n-layer Gaussian weight pyramid. The original image, brightness enhancement image, and contrast enhancement image are input into the n-layer Laplacian image pyramid generation module to obtain an n-layer Laplacian image pyramid. Finally, the n-layer Gaussian weight pyramids of the three images are fused with the n-layer Laplacian image pyramid to obtain the enhanced HSV pixel stream. Finally, the HSV pixel stream is input into the color space conversion module to convert it into an RGB pixel stream. The image fusion generation part includes an image fusion generation module. The n-layer Gaussian and Laplacian pyramid decomposition and reconstruction part includes an n-layer Gaussian image pyramid generation module, an n-layer Laplacian image pyramid generation module, and an image fusion generation module. The image fusion generation part consists of a register-configurable contrast enhancement algorithm, a register-configurable gamma brightness enhancement algorithm, and calculations of grayscale image brightness weights and hue contrast weights. Both the contrast enhancement algorithm and the gamma brightness enhancement algorithm are implemented using hardware circuits with lookup tables. The n-layer Gaussian and Laplacian pyramid decomposition and reconstruction part consists of an image downsampling part, an image upsampling part, an improved mean filtering part, and a block matrix multiplication part, all of which are implemented using hardware circuits in a pipelined manner.

2. The image quality enhancement method based on adjustable parameters according to claim 1, characterized in that... The pipeline buffers the H and S channels after color space conversion using a synchronous FIFO, and then reads them out when proceeding to the image fusion generation module. Simultaneously, the H and S channels are buffered using the synchronous FIFO. When entering the n-layer Gaussian and Laplacian pyramid decomposition and reconstruction section, the i-th layer Gaussian image pyramid is buffered using a synchronous FIFO. After the i-th layer Laplacian image pyramid is calculated, it is read out from the FIFO for subsequent calculations. Once the Gaussian and Laplacian pyramid decomposition and reconstruction section is complete and sum1 is obtained, the H and S channel data are read out synchronously and sent together to the color space conversion module to convert the HSV data stream to an RGB data stream.

3. The image quality enhancement method based on adjustable parameters according to claim 1, characterized in that... The image upsampling part in the n-layer Gaussian and Laplacian pyramid decomposition and reconstruction uses an interlaced zero-padding method and is buffered using an asynchronous FIFO. Each layer of the image pyramid requires an asynchronous FIFO buffer during upsampling. The asynchronous FIFO has a width of 8 bits, a depth 0.5 times the current layer's row resolution, and a read clock four times the write clock. The image downsampling part uses an interlaced pixel extraction method and is also buffered using an asynchronous FIFO. Each layer of the image pyramid requires an asynchronous FIFO buffer during downsampling. The asynchronous FIFO has a width of 8 bits, a depth 0.5 times the current layer's row resolution, and a write clock four times the read clock.

4. The image quality enhancement method based on adjustable parameters according to claim 3, characterized in that... When performing Gaussian pyramid decomposition and reconstruction, filtering is not required for Gaussian pyramid decomposition. However, filtering is only performed after upsampling when performing Laplacian pyramid decomposition. The filtering operation used is an improved mean filter, which selects pixels within the sliding window using a data selector before summing. Only the effective data is averaged, and the sliding window is fixed at 3*3.

Images