Image quality enhancement method based on parameter adjustability

Abstract

The invention belongs to the field of image processing, and provides an image quality enhancement method based on adjustable parameters. The method comprises the steps of firstly converting an input low-illumination and low-contrast RGB image into an HSV image, then obtaining three to-be-fused enhanced images through an image enhancement algorithm, then fusing the three images by using decomposition and reconstruction of an image pyramid, finally converting the fused HSV image into an RGB image, and finally obtaining a high-contrast and low-contrast RGB image. And a high-brightness image is obtained. The fusion algorithm is realized by using a hardware circuit assembly line mode, has real-time performance and adjustability, and is expected to be applied to the fields of security monitoring, night imaging and the like.

Description

Technical Field

[0001] The invention belongs to the field of image processing and relates to an image quality enhancement method based on adjustable parameters. Background Art

[0002] In the actual image capture process, it is common that the captured images are partially or overall darker due to the limited dynamic range of digital cameras. For example, in the case of poor lighting at night, the captured images will show the characteristics of low overall brightness and contrast. Or when shooting in backlight, due to the limitation of exposure settings, it is difficult to capture the brighter background and the darker foreground targets at the same time.

[0003] With the development of society, the demand for high-quality images is increasing day by day, such as in the fields of security monitoring and night imaging. At present, image fusion technology mainly fuses the hue, brightness, color saturation and other aspects of images with different exposure or focus of the same scene, assigns weights to different images, normalizes the weights and then performs weighted summation. However, this method has poor fusion effect in locations where image pixels change dramatically, generally produces artifacts, and requires taking images with different exposures or focuses. It is more complicated to implement and cannot meet the imaging requirements in the fields of security monitoring and night imaging.

[0004] Moreover, the current image fusion is mainly run on the software side, and the image fusion speed cannot meet the real-time requirements.

[0005] Therefore, it is necessary to propose an image quality enhancement method based on adjustable parameters to address the shortcomings of the existing technology. Summary of the invention

[0006] The present invention designs and proposes a parameter-adjustable image quality enhancement method, which can receive a low-illumination, low-contrast image and obtain a high-contrast, high-brightness, high-quality image in real time after hardware circuit processing.

[0007] In order to achieve the above object, the present invention provides an image quality enhancement method based on adjustable parameters, comprising:

[0008] First, the pixel stream of an RGB format image to be enhanced is input into the color space conversion module to convert the RGB pixel stream into an HSV pixel stream.

[0009] Then the HSV image pixel stream is input into the fusion image generation module, which will generate the original image, brightness enhancement image, contrast enhancement image and the weights corresponding to the central pixels of the m*m pixel matrix. The brightness enhancement coefficient and contrast enhancement coefficient as well as the pixel matrix order m and the Gaussian Laplace decomposition layer number n can be input from the outside.

[0010] Then, the weight of the original image, the weight of the brightness enhancement map, and the weight of the contrast enhancement map are input into an n-layer Gaussian image pyramid generation module to obtain an n-layer Gaussian weight pyramid, and the original image, the brightness enhancement map, and the contrast enhancement map are input into an n-layer Laplacian image pyramid generation module to obtain an n-layer Laplacian image pyramid.

[0011] Then, the n-layer Gaussian weight pyramid and the n-layer Laplacian image pyramid of the three pictures are fused to obtain the enhanced HSV pixel stream. During the fusion process, a layered and block processing method is adopted to reduce resource consumption and improve computational efficiency. The specific implementation method is to use the pixel value in the center of the m*m pixel matrix instead of the surrounding pixel values.

[0012] Finally, the HSV pixel stream is input into the color space conversion module to convert the HSV pixel stream into RGB pixel stream.

[0013] Beneficial effects of the invention: The invention designs and proposes an image quality enhancement method based on adjustable parameters. Compared with other image fusion methods, the method has real-time and universality, can complete the enhanced fusion processing of real-time video streams, and is suitable for the fusion enhancement of most low-illumination and low-contrast scenes. It can effectively enhance the dark details of the image while ensuring the image quality. At the same time, the method can be adapted to different hardware environment requirements by adjusting the parameters m and n. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The following drawings are for further illustrating the present invention.

[0015] Figure 1 The hardware architecture diagram of an image quality enhancement method based on adjustable parameters.

[0016] Figure 2 This is the specific architecture diagram of the fused image generation module hdr_derived_input.

[0017] Figure 3 Schematic diagram of the decomposition and reconstruction of a 1920*1080 image using a 2-layer Gaussian pyramid and a Laplacian pyramid.

[0018] Figure 4 These are four cases of improved mean filtering according to the present invention.

[0019] Figure 5 The image to be processed.

[0020] Figure 6 is the processed image. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention more clear, the technical solution of the present invention will be clearly and completely described below in conjunction with the structural schematic diagram of the present invention.

[0022] The hardware architecture of the image quality enhancement method based on adjustable parameters of the present invention is as follows: Figure 1 shown.

[0023] like Figure 1 The leftmost input shown is an RGB color image under weak light conditions. The color space conversion module rgb_hsv separates the brightness, saturation and hue. The separated brightness information will be used to obtain two additional grayscale images through the fusion image generation module hdr_derived_input, which represent the images after brightness enhancement and contrast enhancement respectively, and are output synchronously with the original brightness image. In addition, the hdr_derived_input module will also calculate the weight information of the three grayscale images. The weight information w0, w1, w2 and brightness information g0, g1, g2 output by the module will be input into the Gaussian image pyramid generation module and the Laplace image pyramid generation module as shown in the figure for pyramid decomposition. The n-layer Gaussian weight pyramid and the n-layer Laplace brightness pyramid generated after decomposition will be input into the image fusion module fusion, and each layer will be fused separately and added to the corresponding layers of the other two images to obtain sum1, sum2, sum3, ..., sumn, where Figure 1 Taking n as 3 as an example, sum1, sum2, sum3, …, sumn are input into the image reconstruction module reconstruct for reconstruction to obtain the fused brightness map V. At the same time, the hue H and saturation S obtained after the color space conversion in the first step are input into the delay module delay for synchronous delay. The delayed H, S and brightness map V are input into the color space conversion module hsv_rgb to obtain the fused RGB color image.

[0024] Below Figure 1 Each sub-module is described in detail.

[0025] Color space conversion module rgb_hsv. Let (r, g, b) be the three color components of red, green and blue, and their value range is [0, 1]. Let max be the maximum value of r, g, b, and min be the minimum value. The (h, s, v) value in the HSV space is required. Here, the range of hue h is [0, 360), and the range of saturation s and brightness l is [0, 1]. The calculation formula is:

[0026]

[0027]

[0028] The fused image generation module hdr_derived_input has the following hardware architecture: Figure 2 shown.

[0029] The fused image generation module is used to obtain brightness-enhanced and contrast-enhanced images from grayscale images, and add the original grayscale images to output a total of three brightness images for subsequent fusion. At the same time, the module also outputs weight images corresponding to the three grayscale images.

[0030] Figure 2 The contrast enhancement in the image adopts a linear contrast enhancement algorithm to improve the image contrast, and the brightness enhancement adopts a gamma brightness enhancement algorithm. Both the contrast and brightness can be configured by passing parameters through registers. The contrast of the output image is adjusted accordingly according to the contrast difference of the input image. If the contrast of the input image is poor, the contrast of the output image can be increased by changing the parameters. If the contrast of the input image is good, the contrast of the output image is slightly adjusted or not adjusted. The same is true for the brightness enhancement algorithm.

[0031] Next, the brightness weight and hue contrast weight are calculated for the three grayscale images. Pixels with good exposure have higher weights, and pixels with overexposure or underexposure have lower brightness weights. The calculation formula here mainly considers that various images in the real world basically satisfy the Gaussian distribution. In simple terms, the position with a pixel value of 128 generally has a higher brightness weight. The contrast weight should not only consider the brightness information, but also the color information. Therefore, the three HSV channels need to be used for calculation at the same time. The calculation formulas for brightness weight and hue contrast weight are as follows:

[0032]

[0033] W c (x,y)=I(x,y)(1+cos(αH(x,y)+φ)S(x,y))

[0034] The weight calculation method of the present invention uses a lookup table, where W b (x, y) is the brightness weight of pixel I(x, y), W c (x,y) is the hue contrast weight of pixel I(x,y), H(x,y) is the hue of pixel I(x,y), S(x,y) is the saturation of pixel I(x,y), and αφ is an adjustable parameter.

[0035] In the hdr_weight_normalize module, the brightness weight and contrast weight of each image are multiplied, and then the weights of the three images after multiplication are normalized and output separately. The formula is as follows.

[0036] W(x,y)=Wb (x,y)W c (x,y)

[0037]

[0038] in is the normalized weight of the k-th image output, W k (x,y) is the weight of the kth image.

[0039] Gaussian and Laplace pyramid decomposition and reconstruction modules hdr_gaussian_pyramid_nlevel, hdr_laplacian_py ramid_nlevel, fusion. Here, the decomposition and reconstruction of a 1920*1080 image using a 2-layer Gaussian pyramid and a Laplace pyramid are taken as an example. Figure 3 shown.

[0040] Figure 3 The weight 1920x1080 on the left is the weight map, and the gray 1920x1080 is the brightness map. The weight map is decomposed by Gaussian to obtain the second-level Gaussian pyramid gaussian2960x540. The original image is delayed by two rows of pixels to obtain the first-level Gaussian pyramid gaussian11920x1080. The brightness map is decomposed by Laplacian to obtain the second-level Laplacian pyramid laplacian2960x540. The second-level Laplacian pyramid is upsampled, filtered, and then subtracted from the original image to obtain the first-level Laplacian pyramid.

[0041] Then, the first and second layers of the Gaussian pyramid and the Laplacian pyramid are multiplied to obtain sum2960x540 and sum11920x1080 respectively, sum2960x540 is upsampled, filtered to obtain up_sum21920x1080, and finally summed with sum11920x1080 to obtain the fused and reconstructed brightness image. In the process of multiplying the Gaussian pyramid and the Laplacian pyramid, a block method is adopted, and the size of the block matrix m is configured through a register. If the image resolution is large, m can be set larger, and the operand isolation technology is adopted to set the enable signal, which can save computing resources and reduce computing power consumption. If the image resolution is small, m can be set to 1, that is, no block is performed.

[0042] In addition, this is just the decomposition and reconstruction of a brightness image. Two other images must be added at the position of mult to get a complete image. The following is an introduction to the algorithms used in each module:

[0043] Downsampling module: This downsampling design uses the method of extracting pixels in alternate rows and columns, retaining only the pixels in odd rows and columns, and uses an 8-bit wide and 512 (should be 0.5 times the row resolution of the current layer image) deep asynchronous FIFO as a cache. The frequency of the write clock should be 4 times that of the read clock.

[0044] Upsampling module: This design uses the method of padding every other row and column with zeros. It also uses an 8-bit wide and 512-bit (should be 0.5 times the row resolution of the current layer image) deep asynchronous FIFO as the cache. The read clock frequency is 4 times the write clock frequency.

[0045] Improved mean filtering proposed in the present invention: After verification, it was found that this design can be directly down-sampled without filtering, which has little effect on the decomposition and reconstruction of the image. Therefore, the filtering operation before down-sampling can be optimized, and filtering is only performed after up-sampling. Since up-sampling uses zero padding for alternate rows and columns, if ordinary mean filtering is used directly, the image brightness will be reduced. Therefore, it is improved by selecting only valid data for averaging. In order to save resources, a 3*3 sliding window is still used. There will be four situations as follows Figure 4 As shown, the filtering operation can be completed as long as these four situations are specially processed.

[0046] Color space conversion module hsv_rgb, the conversion formula from HSV to RGB is:

[0047]

[0048] For each color vector (r,g,b).

[0049]

[0050] Figure 5 , Figure 6 The figure shows the comparison of the effects after being processed by the parameter-adjustable image quality enhancement method. It can be clearly seen that after the image is processed, its brightness, contrast and color are significantly improved while ensuring the quality.

[0051] The above only expresses the implementation mode of the present invention, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A method for enhancing image quality based on adjustable parameters, characterized in that: A fused image generation part and an n-layer Gaussian and Laplace pyramid decomposition and reconstruction part; the fused image generation part is composed of a register-configurable contrast enhancement algorithm, a register-configurable gamma brightness enhancement algorithm, and calculation of grayscale image brightness weight and hue contrast weight, wherein the contrast enhancement algorithm and the gamma brightness enhancement algorithm are both implemented using hardware circuits in a lookup table manner, and can be replaced by any other algorithm that is easy to implement; the n-layer Gaussian and Laplace pyramid decomposition and reconstruction part is composed of an image downsampling part, an image upsampling part, an improved mean filtering part, and a block matrix multiplication part, all of which are implemented using hardware circuits in a pipeline manner.

2. The assembly line according to claim 1, characterized in that It uses a synchronous fifo to cache the H and S channels after the color space conversion, and reads them out when it reaches the fusion image generation module, and continues to cache the H and S channels using the synchronous fifo; when entering the n-layer Gaussian and Laplace pyramid decomposition and reconstruction part, the i-th layer of the Gaussian image pyramid is cached using the synchronous fifo, and then waits until the i-th layer of the Laplace image pyramid is calculated before reading the i-th layer of the Gaussian image pyramid from the fifo for subsequent calculations; when the Gaussian and Laplace pyramid decomposition and reconstruction part is completed and sum1 is obtained, the data of the H and S channels are read out synchronously and sent to the color space conversion module to convert the HSV data stream into an RGB data stream.

3. According to the n-layer Gaussian and Laplacian pyramid decomposition and reconstruction part according to claim 1, it is characterized in that The image upsampling part used in the decomposition and reconstruction of the n-layer Gaussian and Laplacian pyramid uses the method of padding zeros in alternate rows and columns, and uses asynchronous fifo for buffering. Each layer of the image pyramid needs to use asynchronous fifo for buffering when upsampling. The width of the asynchronous fifo is 8 bits, the depth of the asynchronous fifo is 0.5 times the row resolution of the current layer image, and the read clock of the asynchronous fifo is 4 times the write clock; the image downsampling part uses the method of extracting pixels in alternate rows and columns, and uses asynchronous fifo for buffering. Each layer of the image pyramid needs to use asynchronous fifo for buffering when downsampling. The width of the asynchronous fifo is 8 bits, the depth of the asynchronous fifo is 0.5 times the row resolution of the current layer image, and the write clock of the asynchronous fifo is 4 times the read clock.

4. The n-layer Gaussian and Laplacian pyramid decomposition and reconstruction part according to claim 3 is characterized in that No filtering operation is required when performing Gaussian image pyramid decomposition; when performing Laplacian pyramid decomposition, filtering is performed only after upsampling, and the filtering operation adopted is an improved mean filter, which uses a data selector to select pixels in the sliding window and then sum them, only the valid data is averaged, and the sliding window is fixed to 3*3.

Images