A high dynamic image enhancement method

Abstract

The present invention discloses a high-dynamic image enhancement method, which is mainly used for the enhanced processing and display of ultra-high dynamic images in real-time image processing systems. The histogram equalization algorithm can effectively compress the dynamic range of high-dynamic images and improve the contrast of high-dynamic images, but directly performing histogram equalization on ultra-high dynamic images occupies too much memory, and ordinary embedded hardware usually cannot meet the memory and real-time processing requirements. In view of this problem, the present invention proposes an improved method, which utilizes a two-level segmented histogram equalization method to greatly reduce memory usage, so that the algorithm's enhanced processing effect on ultra-high dynamic images is close to that of conventional histogram equalization. The present invention has the characteristics of fast computing speed, no parameter setting, and small memory usage, and can be applied to real-time ultra-high dynamic image hardware processing systems.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a high dynamic image enhancement method. Background Art

[0002] At present, high dynamic range image display has a wide range of applications, such as image recording and display of driving recorders under strong light at night, display of high dynamic welding images by industrial welding robots, etc. The brightness dynamic range of the image is large, and the number of image bits is high, which can reach 20 bits or more. However, most display devices can only display 8-bit images, so special algorithms are needed to process such data above 20 bits so that it can retain the visual effect of the original scene in the human eye. The purpose of high dynamic image enhancement technology is to compress the dynamic range of high dynamic images so that they can be displayed on display devices with low dynamic range and can be used for subsequent image processing. The existing high dynamic image enhancement technology is difficult to apply on embedded platforms due to its complex calculation. Histogram equalization is an image enhancement algorithm with good effect, but for ultra-wide dynamic images of 20 bits or higher, its histogram is too large, and the calculation amount and storage amount are large, and general chips are difficult to meet the requirements. Therefore, for the existing technology, it is difficult for the current high dynamic image enhancement algorithm to realize real-time enhancement processing of high dynamic images on embedded systems. Summary of the invention

[0003] The present invention proposes a real-time enhancement processing algorithm for high-dynamic video images based on a low-cost embedded hardware platform, which has the characteristics of small memory occupation and low computational complexity.

[0004] The objective of the present invention is achieved through the following technical solutions:

[0005] A high dynamic image enhancement algorithm comprises the following steps:

[0006] S1: Take the high-order histogram H of the high-order K1-bit statistical image of the input image f0 ;

[0007] S2: According to the histogram H f0 Calculate the histogram equalization transformation function M0;

[0008] S3: Use histogram equalization transformation function M0 to transform the histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ;

[0009] S4: In H g0 Find the n zero-value intervals with the largest length and sort them from long to short according to the length of the zero-value intervals as G1, G2, …G n ; or in H g0Find the n zero value intervals on the left corresponding to the largest n values, and sort them from large to small as G1, G2, ... G n ; Each zero value interval includes all consecutive zero values ​​and the first non-zero value on the right;

[0010] S5: For zero value interval G i , i=1,2,…n respectively find the grayscale interval endpoints of the input image and their corresponding grayscale interval endpoints of the output image;

[0011] S6: The zero value intervals G of the input image found i , i=1,2,…n, linearly transform to K2 bits respectively, and count the histogram H of each interval respectively f1 ,H f2 ,…,H fn ;

[0012] S7: Using each histogram H fi , i = 1, 2, ... n and their corresponding linear transformation functions, calculate their corresponding histogram equalization transformation functions M i ,i=1,2,…n;

[0013] S8: Use each histogram equalization transformation function to transform the grayscale value of each pixel of the input image.

[0014] When the input image is an RGB color image, convert the image to the HSV color space and only process the V component.

[0015] When the input image is an RGB color image, the image is converted to the HSI color space and only the I component is processed.

[0016] The present invention also discloses a high dynamic video image enhancement method, which performs frame-by-frame image enhancement on an input video, and is characterized by comprising the following steps:

[0017] S11: Take the high-order histogram H of the high-order K1-bit statistical image for each frame of input image f0 ;

[0018] S12: Use the high-order histogram H of the previous frame image f0 Calculate the histogram equalization transformation function M0;

[0019] S13: Use the above histogram equalization transformation function M0 to transform the histogram H f0 Perform grayscale transformation and output the transformed histogram H g0 ;

[0020] S14: In H g0Find the n zero-value intervals with the largest length, and sort them from long to short or from large to small according to the length of the zero-value interval or the adjacent non-zero values ​​on the right. n ; or in H g0 Find the n zero value intervals on the left corresponding to the largest n values, and sort them from large to small as G1, G2, ... G n ;

[0021] S15: For interval G i , i=1,2,…n respectively find the grayscale interval endpoints of the input image and their corresponding grayscale interval endpoints of the output image;

[0022] S16: The found intervals G of the input image are i , i=1,2,…n, linearly transform the grayscale to K2 bits respectively, and count the histogram H of each interval of the current frame image respectively f1 ,H f2 ,…,H fn ;

[0023] S17: Using each histogram H fi , i = 1, 2, ... n, calculate the corresponding histogram equalization transformation function M i ,i=1,2,…n;

[0024] S18: According to the histogram equalization transformation function M i , i=1,2,…n, and the transformation function M0 obtained in the previous frame, transform the grayscale value of each pixel of the input image of the next frame to obtain the output image.

[0025] The beneficial effect of the present invention is that the high dynamic graphics enhancement method of the present invention occupies small memory and has low computational complexity, and can use low-cost FPGA hardware devices or embedded processors to implement a high dynamic image histogram equalization enhancement processing algorithm, thereby realizing real-time enhancement processing of ultra-high dynamic images. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a flow chart of the high dynamic image enhancement method in the present invention.

[0027] Figure 2 It is a flow chart of the high dynamic video image enhancement method in the present invention.

[0028] Figure 3 It is the histogram Hg0 and its first 4 zero value intervals in the specific embodiment, and its corresponding relationship with the high 13-bit image and 20-bit image values. DETAILED DESCRIPTION

[0029] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation examples. The following examples are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0030] Embodiment 1:

[0031] For the input high dynamic image, the flowchart of the specific enhancement method is as follows Figure 1 As shown, the specific process is as follows:

[0032] S1. Take the high-order histogram H of the high-order K1-bit statistical image of the input image f0 ;

[0033] If the input image is in a color space without a brightness component, convert it to a color space such as HSV or HSI and take the brightness component (V or I) of the image. The formula for converting the V component to HSV is as follows:

[0034] V=max(R,G,B)

[0035] Where R, G, B represent the components of each pixel of the high dynamic image in the red, green, and blue channels respectively, and V represents the brightness component of the input image.

[0036] If the input image is a monochrome image, only the brightness component V is included.

[0037] Then, for the brightness component V of the input image, take the high K1 bits of its binary number to get the high bit V of the brightness component of the input image scaled :

[0038] V scaled =V>>(K max -K1)

[0039] Where K max It is the number of binary quantization bits corresponding to the maximum grayscale value in the input high dynamic image, or the number of quantization bits of the input image.

[0040] Then count the quantized brightness component V of the input image scaled The high-order histogram H f0 The histogram H f0 It is also called a first-level histogram. The specific formula is as follows:

[0041]

[0042] Where m and n are the width and height of the image respectively, and the delta function is defined as:

[0043]

[0044] S2. Use the histogram H f0Calculate the histogram equalization transformation function M0. The mapping function M0(k) is calculated as follows:

[0045]

[0046] Where T is the total number of pixels in the image, and N is the highest grayscale level of the display device, usually 255 or 256.

[0047] S3. Use histogram equalization transformation function M0 to transform the histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ;

[0048] First traverse the histogram H f0 , each grayscale value of the histogram is mapped to a new grayscale value according to M0, and the transformed histogram H is obtained g0 , due to H g0 The value range is 0~N, H f0 The value range is K>8, so H will appear f0 When different grayscales are mapped to the same new grayscale value, the corresponding pixel numbers can be merged, that is, they are added together as the histogram value of the new grayscale value.

[0049] S4. In H g0 Find the n zero-value intervals with the largest length and sort them from long to short according to the length of the zero-value intervals as G1, G2, ...G n ; or in H g0 Find the n zero value intervals on the left corresponding to the largest n values, and sort them from large to small as G1, G2, ...G n ; Finding the n zero-value intervals with the largest length means finding a preset number of n zero-value intervals starting from the maximum length; finding the largest n values ​​means finding a preset number of n values ​​starting from the maximum value.

[0050] Specifically, start from 0 to traverse the histogram H g0 , that is, traverse the histogram H from left to right g0 , start recording the interval when the number of pixels in the histogram is 0, and continue until the number of pixels is not 0, record the interval position and the length of the interval; another method is to record the interval position and the length of the interval in the histogram H g0 First find the largest n values ​​with 0 on the left, record the interval position and the length of the interval or the value. Such intervals are called zero value intervals, and each zero value interval includes all consecutive zero values ​​and the first non-zero value on the right.

[0051] After traversing the entire histogram, several intervals with zero pixel points are obtained, and then the intervals are sorted from long to short according to the recorded interval length or from large to small according to the corresponding right value.

[0052] For example, the interval found is:

[0053] [G il , G ir ], i = 1, 2, ..., n

[0054] Among them G i1 , G ir Respectively represent the left and right endpoints of the interval, G ir Corresponding to the first non-zero value on the right, G i1 Corresponds to the initial zero value on the left side (from left to right).

[0055] like Figure 3 Shown is the histogram H of an image g0 The interval example is as follows: the input image is a 20-bit image, and the upper 13 bits (K1=13) are taken to calculate the high-bit histogram.

[0056] S5. For zero value interval G i , i=1, 2, ...n respectively find the grayscale interval endpoints of the input image and their corresponding grayscale interval endpoints of the output image;

[0057] Suppose some interval [G il , G ir ], its G il and G ir Corresponding to the original image histogram H f0 The minimum and maximum grayscale values ​​are V i，min 、V i，max One method is to determine the interval endpoint value according to the grayscale mapping table. First, the interval endpoint value in the output image can be determined according to the value in the grayscale mapping table. Another method is to simultaneously record H in step S3. g0 The original histogram H corresponding to each gray value f0 The gray value range of the i-th interval [G il , G ir ] corresponds to H f0 Since only the high K1 bits of the image are input during the first equalization, the histogram H is obtained by counting the K1 bits of the grayscale value of the input image after it is right-shifted. f0 , so corresponding to H f0 Shifting the index to the left by the corresponding number of bits can determine the grayscale value range of the input image corresponding to the interval endpoint [V i，min , V i，max ] The specific mapping relationship between the interval endpoints of an example image is as follows: Figure 3As shown, 0 directly corresponds to the minimum value 0 on the 20-bit image; 65 corresponds to the value of 0 on the high 13-bit image, and the maximum value on the 20-bit image is (0+1)*2 7 -1=127; 66 corresponds to 1 for the high 13-bit image, and the minimum value for the 20-bit image is 1*2 7 =128; 138 corresponds to the value of 1 in the high 13-bit image, and the maximum value on the corresponding 20-bit image is (1+1)*2 7 -1=255;...;211 corresponds to 10 for the high 13-bit image, and the minimum value for the 20-bit image is 10*2 7 =1280; 255 corresponds to the value of the high 13-bit image is 8191, and the maximum value of the corresponding 20-bit image is (8191+1)*2 7 -1 = 1,048,575. The interval [3,9] of the upper 13-bit image is merged into the 8-bit image after histogram equalization, and the interval [10,8191] of the upper 13-bit image is transformed into the interval [211,255] of the 8-bit image.

[0058] S6. The zero value intervals G of the input image found i , i = 1, 2, ... n, linearly transform to K2 bits respectively, and count the histogram H of each interval respectively f1 , H f2 , ..., H fn ;

[0059] Perform linear transformation on each grayscale interval of the original image so that the i-th interval [V i，min , V i，max ] maps to K2 is the number of binary bits corresponding to the range of the statistical histogram, which determines the size of the histogram. This histogram is called a secondary histogram. K2 can be the same as K1. i，min , V i，max ] interval is linearly transformed, and [V i，min , V i，max ] interval transformation to The specific linear transformation method is:

[0060]

[0061] Where V is [V i，min , V i，max ] The grayscale value of the original image in the interval.

[0062] Then, the histogram H of y after grayscale transformation of each grayscale value interval of the original image is counted respectively f1 , H f2 , ..., H fn .

[0063] Figure 3 In the example, for n=4 longest non-zero intervals, find the 4 corresponding value intervals of the original 20-bit image: [0, 127], [128, 255], [256, 383] and [384, 1279].

[0064] S7. Using each histogram H fi , i = 1, 2, ... n calculate the corresponding histogram equalization transformation function M i , i=1,2,...n,

[0065] The calculation formula is as follows:

[0066] i=1,2,...,n

[0067] Where N i is the output image interval length: N i =T ir -T il , where T i is the total number of pixels in the region: This value is equal to the first non-zero value on the right side of the corresponding zero value interval on the high-order histogram; wherein y is calculated according to the formula in step S6 based on the interval.

[0068] S8. Use each histogram equalization transformation function to transform the grayscale value of each pixel of the input image:

[0069]

[0070] Where g(x,y) is the grayscale value after transformation in each interval.

[0071] If the input image is a color image, the grayscale value after transformation and H and S in the HSV or HSI space obtained in step S1 can be used to transform it back to the RGB color space or other color space.

[0072] In this embodiment, if a conventional histogram equalization algorithm is used to process a 20-bit ultra-high dynamic image, the histogram size is more than 1 million units, and the memory required is 8MB, which not only occupies a large amount of memory, but also takes a long time to clear the histogram, making it difficult to achieve real-time video processing; if the method of the present invention is used, the number of intervals is set to 8, K1 is 13, and K2 is 13, then the memory occupied is 360KB. Therefore, the method can realize real-time ultra-high dynamic image histogram equalization processing on a low-cost FPGA or embedded processor.

[0073] The present invention approaches the processing effect of the conventional histogram algorithm through secondary histogram equalization, so the conventional histogram equalization processing result can be used as a benchmark, and the degree of approximation of the processing effect of the present invention on high dynamic images to the processing effect of the conventional histogram equalization algorithm is used as the performance indicator of the present invention. The results of PSNR calculation are specifically shown in Table 1, which shows the degree of approximation of the processing effect of the present invention on 7 high dynamic images to the processing effect of the conventional histogram equalization algorithm. When the value of K1 increases, the PSNR value also increases, and when K1 is 13 bits or more, the approximation effect on these images is better.

[0074] Table 1 PSNR values ​​of the algorithm relative to the common histogram equalization in various scenarios (unit: dB)

[0075]

[0076] Embodiment 2:

[0077] The input high dynamic video is subjected to histogram equalization enhancement processing frame by frame. The flowchart of the specific enhancement method is as follows: Figure 2 As shown, the main difference from the first embodiment is that the current frame image is processed according to the high-order histogram of the previous frame statistics and the secondary histogram of the previous frame statistics. The specific process is as follows:

[0078] S11: Take the high-order histogram H of the high-order K1-bit statistical image for each frame of input image f0 .

[0079] High dynamic video inputs image data frame by frame. It is difficult to store a frame of image data for direct processing in the hardware processing system. Therefore, the video data can be processed in a pipeline by taking advantage of the similar content of adjacent frames of the video, and the result information calculated in the previous frame of the image is used for the processing of the next frame. The high K1-bit histogram H is counted for each frame of the input system. f0 ,The specific steps are the same as S1.

[0080] S12: Use the high-order histogram H of the previous frame image f0 Calculate the histogram equalization transformation function M0.

[0081] The specific operation of calculating the transformation function M0 is the same as S2.

[0082] S13: Use the above histogram equalization transformation function M0 to transform the histogram H f0 Perform grayscale transformation and output the transformed histogram H g0 The specific operation is the same as S3.

[0083] S14: In H g0Find the n zero-value intervals with the largest length, and sort them from long to short or from large to small according to the length of the zero-value interval or the adjacent non-zero values ​​on the right. n ; or in H g0 Find the n zero value intervals on the left corresponding to the largest n values, and sort them from large to small as G1, G2, ... G n ; The specific operation of this step is the same as step S4.

[0084] S15: For interval G i , i=1,2,…n respectively find the grayscale interval endpoints of the input image and their corresponding grayscale interval endpoints of the output image; the specific operation of this step is the same as step S5.

[0085] In practical application, the high-order histogram H is obtained in step S11. f0 After that, steps S12, S13, S14 and S15 can be immediately executed in the incoming frame blanking period for the next frame processing. That is, when the current frame image arrives, the n zero value intervals in order have been found according to the high-order histogram of the previous frame image, and the left and right endpoints of each interval have been found.

[0086] S16: The found intervals G of the input image are i , i=1,2,…n, linearly transform the grayscale to K2 bits respectively, and count the histogram H of each interval of the current frame image respectively f1 ,H f2 ,…,H fn The specific operation of this step is basically the same as step S6, except that the grayscale interval is obtained based on the high-order histogram of the previous frame image.

[0087] If the previous frame image is recorded as the first frame, the current frame image is called the second frame image.

[0088] When the current frame image is input, each interval and its left and right endpoints have been calculated based on the previous frame image. Therefore, it is possible to determine which interval the grayscale of the input pixel belongs to based on this endpoint, and generate a histogram of the input pixel according to different intervals.

[0089] S17: Using each histogram H fi , i = 1, 2, ... n, calculate the corresponding histogram equalization transformation function M i ,i=1,2,…n;

[0090] After the histogram statistics of each interval are completed in step S16, the histogram equalization transformation function of each interval can be calculated according to the histogram information of the interval, and the specific operation is the same as step S7. This step is completed in the blanking period after the second frame image.

[0091] S18: According to the histogram equalization transformation function M i ,i=1,2,…n(M i The grayscale value of each pixel of the input image of the next frame (which can be recorded as the third frame) is transformed by using the transformation function M0 obtained in the previous frame (i.e., the first frame) (M0 is the first-level transformation function) and the transformation function M1 obtained in the previous frame (i.e., the first frame) to obtain the output image. The specific operation is basically the same as step S8, except that the transformation function M1 obtained in the first frame and the transformation function M2 obtained in the first frame are used for the histogram equalization transformation of the third frame image. i ,i=1,2,…n.

[0092] When the third frame image is input into the system, the system has already calculated the grayscale mapping table for each interval. Therefore, when the pixels of the third frame image are input into the system, it is possible to determine which grayscale interval the pixel belongs to based on its grayscale, thereby selecting the corresponding grayscale mapping table, and then obtaining the grayscale mapping result based on the corresponding grayscale mapping table.

[0093] The above process processes each frame of the image in a loop. Except for the first two frames input, the processing results of other images can be output in real time with almost no delay.

[0094] The description of the specific embodiments described above is only applicable to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made to the present invention without departing from the principle of the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A high dynamic image enhancement method, characterized in that: The following steps are involved: S1: Take the high-order histogram H of the high-order K1-bit statistical image of the input image f0 ; S2: According to the histogram H f0 Calculate the histogram equalization transformation function M0; S3: Use histogram equalization transformation function M0 to transform the histogram H f0 Perform grayscale transformation to obtain the transformed histogram H g0 ; S4: In H g0 Find the n zero-value intervals with the largest length and sort them from long to short according to the length of the zero-value intervals as G1, G2, …G n ; or in H g0 Find the n zero value intervals on the left corresponding to the largest n values, and sort them from large to small as G1, G2, ... G n ; Each zero value interval includes all consecutive zero values ​​and the first non-zero value on the right; S5: For zero value interval G i , i=1,2,…n respectively find the grayscale interval endpoints of the input image and their corresponding grayscale interval endpoints of the output image; S6: The zero value intervals G of the input image found i , i=1,2,…n, linearly transform to K2 bits respectively, and count the histogram H of each interval respectively f1 ,H f2 ,…,H fn ; S7: Using each histogram H fi , i = 1, 2, ... n and their corresponding linear transformation functions, calculate their corresponding histogram equalization transformation functions M i ,i=1,2,…n; S8: Use each histogram equalization transformation function to transform the grayscale value of each pixel of the input image.

2. The high dynamic image enhancement method according to claim 1, characterized in that: In step S1, when the input image is an RGB color image, the image is converted into the HSV color space and only the V component is processed.

3. The high dynamic image enhancement method according to claim 1, characterized in that: In step S1, when the input image is an RGB color image, the image is converted into an HSI color space and only the I component is processed.

4. A high dynamic video image enhancement method, which performs frame-by-frame image enhancement on an input video, characterized in that The steps include: S11: Take the high-order histogram H of the high-order K1-bit statistical image for each frame of input image f0 ; S12: Use the high-order histogram H of the previous frame image f0 Calculate the histogram equalization transformation function M0; S13: Use the above histogram equalization transformation function M0 to transform the histogram H f0 Perform grayscale transformation and output the transformed histogram H g0 ; S14: In H g0 Find the n zero-value intervals with the largest length, and sort them from long to short or from large to small according to the length of the zero-value interval or the adjacent non-zero values ​​on the right. n ; or in H g0 Find the n zero value intervals on the left corresponding to the largest n values, and sort them from large to small as G1, G2, ... G n ; S15: For interval G i , i=1,2,…n respectively find the grayscale interval endpoints of the input image and their corresponding grayscale interval endpoints of the output image; S16: The found intervals G of the input image are i , i=1,2,…n, linearly transform the grayscale to K2 bits respectively, and count the histogram H of each interval of the current frame image respectively f1 ,H f2 ,…,H fn ; S17: Using each histogram H fi , i = 1, 2, ... n, calculate the corresponding histogram equalization transformation function M i ,i=1,2,…n; S18: According to the histogram equalization transformation function M i , i=1,2,…n, and the transformation function M0 obtained in the previous frame, transform the grayscale value of each pixel of the input image of the next frame to obtain the output image.

Images