Infrared image enhancement method combining histogram equalization and bilateral filtering

Abstract

The invention discloses an infrared image enhancement method combining histogram equalization and bilateral filtering, which comprises the following steps of: 1, acquiring an infrared video, and obtaining an original infrared image; 2, segmenting the original infrared image to obtain a foreground region and a background region; 3, histogram equalization of weighted gray values is carried out on the foreground region; 4, performing contrast-limited histogram equalization on the background region; 5, superposing the foreground region and the background region to obtain a fused image; 6, performing bilateral filtering noise reduction on the fused image; and 7, obtaining and outputting a final enhanced image. According to the method, the visual effect of the infrared image can be effectively improved, and the problems of over-amplification of noise, insufficient enhancement of a target, unclear edge texture, over-darkness of the image and the like are solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to an infrared image enhancement method combining histogram equalization and bilateral filtering. Background Art

[0002] Infrared imaging technology is a technology that uses the radiation of the target object to form an image. It is widely used in military and civilian fields such as weapon guidance, industrial detection, and assisted driving. Due to the limitations of the imaging mechanism and manufacturing process of the detector, compared with visible light images, infrared images have low contrast, high noise, and a narrow grayscale range, making subsequent detection and tracking more difficult. Therefore, it is of great significance to improve the quality of infrared images.

[0003] Among traditional methods, histogram equalization is a commonly used image enhancement technology. However, the traditional histogram equalization method will cause the background noise to be over-amplified, and the details of the foreground will be hidden or even lost. In order to overcome the defects of histogram equalization, many improved methods have been proposed. For example, Vickers proposed a histogram platform equalization method, Kun Liang proposed a dual-platform histogram equalization method, and Zuiderveld proposed a contrast-limited adaptive histogram equalization. Existing methods can improve image quality to a certain extent, but in fields such as military reconnaissance, the target is far away from the camera, and the overall imaging is dark. When using the traditional histogram equalization method, there are still problems such as over-amplification of noise, under-enhancement of the target, unclear edge texture, and too dark image. Summary of the invention

[0004] In view of the shortcomings of the traditional histogram equalization image enhancement method, the present invention proposes an infrared image enhancement method combining histogram equalization and bilateral filtering, which can effectively improve the visual effect of infrared images and solve the problems of over-amplification of noise, under-enhancement of targets, unclear edge textures, and too dark images.

[0005] The technical solution for implementing the present invention is as follows:

[0006] An infrared image enhancement method combining histogram equalization and bilateral filtering comprises the following steps:

[0007] Step 1: Collect infrared video and obtain original infrared image;

[0008] Step 2: Segment the original infrared image to obtain the foreground area and the background area;

[0009] Step 3: Perform histogram equalization of weighted grayscale values ​​on the foreground area;

[0010] Step 4: Perform restricted contrast histogram equalization on the background area;

[0011] Step 5: Superimpose the foreground area and the background area to obtain a fused image;

[0012] Step 6: Perform bilateral filtering to reduce noise on the fused image;

[0013] Step 7: Get the final enhanced image and output it.

[0014] Furthermore, in step 2, threshold segmentation is used to roughly separate the foreground and background, and then a clustering algorithm is used to perform a finer segmentation, and finally morphological operations are used to remove noise and fill the target area to make the area contour more complete; thereby obtaining the foreground area and background area of ​​the original infrared image.

[0015] Furthermore, in step three, the local variance of the grayscale of pixels in the foreground area is calculated one by one, the weight of the grayscale of the pixel is obtained according to the local variance of the grayscale of the pixel, the grayscale value of each pixel is multiplied by the weight to obtain the weighted grayscale value of each pixel, and a histogram is constructed using the weighted grayscale values ​​of each pixel in the foreground area. Finally, the histogram is equalized to obtain an enhanced image of the foreground area.

[0016] Furthermore, in step 4, the background area is divided into n*n sub-blocks of equal size and non-overlapping, the histogram of each sub-block is calculated, and the adaptive clipping amplitude T is set according to the uniformity of each sub-block. A lower clipping amplitude is set if the sub-block is uniform, and a higher clipping amplitude is set if the sub-block is uneven; the histogram is clipped according to the clipping amplitude, and the pixels are redistributed; the redistributed sub-block histogram is equalized, and finally an interpolation operation is performed to obtain a new pixel value;

[0017] The uniformity β is expressed by the grayscale average value Avg and the grayscale standard deviation σ;

[0018] β=σ / Avg

[0019] If the uniformity of the sub-block is less than that of the entire background area, then according to the pixel distribution of the sub-block histogram, the gray level of the current 10% of the pixels is selected, and the corresponding pixel numbers are pix1, pix2, ..., pix g , g is the number of gray levels in the first 10%, and the clipping amplitude is obtained by the following formula;

[0020]

[0021] If the uniformity of the sub-block is greater than that of the entire background area, then according to the pixel distribution of the sub-block histogram, the gray level of the current 20% of the pixels is selected, and the corresponding pixel numbers are pix1, pix2, ..., pix h , h is the number of gray levels in the first 20%, and the clipping amplitude is obtained by the following formula;

[0022]

[0023] After the clipping amplitude is obtained, the histogram is clipped according to the clipping amplitude. In the process of redistributing the pixels, the total number of pixels exceeding the clipping amplitude is evenly distributed to each gray level that is not zero.

[0024] Furthermore, in step five, since the foreground and background use different histogram equalization methods, the contour at the regional seam is too strong during fusion, and the grayscale of the foreground area and the background area is adjusted before superposition.

[0025] Beneficial effects:

[0026] 1. The present invention divides the original infrared image into a foreground area and a background area, performs grayscale weighted improved histogram equalization on the foreground area, performs improved limited contrast histogram equalization on the background area, fuses the foreground area with the background area, and performs bilateral filtering denoising on the fused image, which can solve the problems of over-amplification of noise, under-enhancement of targets, unclear edge textures, and too dark images.

[0027] 2. In the equalization process of the background area, the present invention sets the adaptive clipping amplitude according to the uniformity of each sub-block, so that each sub-block can have a better image enhancement effect.

[0028] 3. The enhanced image of the present invention has better visual effect and can lay a good foundation for subsequent detection, tracking, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of the present invention.

[0030] Figure 2 This is a schematic diagram of the improved cutting and allocation of the present invention.

[0031] Figure 3 The figures are comparison diagrams of the treatment results of the present invention; (a) is before treatment, and (b) is after treatment. DETAILED DESCRIPTION

[0032] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0033] A method for infrared image enhancement combining histogram equalization and bilateral filtering is described in detail as follows: Figure 1 shown.

[0034] Step S1: Capture infrared video through an image acquisition device such as an infrared camera, perform frame operations on the video, and obtain an original infrared image I.

[0035] Step S2: Segment the original infrared image to obtain a foreground area and a background area. According to the method of the present invention, by analyzing the characteristics of the infrared image, due to the different visual characteristics of the foreground and background of the infrared image, the foreground area contains more information and the background area contains less information. In order to obtain a better segmentation effect, the foreground and background are roughly separated by threshold segmentation, and then a clustering algorithm is used for more refined segmentation. Finally, morphological operations are used to remove noise and fill the target area to make the area contour more complete. After the above operations, the foreground area I1 and the background area I2 of the original infrared image I can be obtained.

[0036] Step S3: Perform grayscale weighted improved histogram equalization on the foreground area. First, calculate the local variance Var(x,y) of the pixels in the foreground area I1 one by one. The calculation formula of the local variance is as follows:

[0037]

[0038] Where I1(x+i,y+j) represents the grayscale value of the pixel in the window centered at (x,y); the window size is (2k+1)×(2k+1), and k is 2 in this embodiment; N s Indicates the number of pixels in the window. In this embodiment, N s is 25; M(X) represents the mean grayscale value of the local window pixels. The calculation formula of the mean is:

[0039]

[0040] Then, the weight of the pixel grayscale is obtained according to the local variance of the pixel grayscale (x, y). The weight calculation formula of the pixel grayscale is:

[0041]

[0042] In the formula, is the sum of the local variances of the grayscale of all pixels in the foreground area. Multiply the grayscale value of each pixel by the weight to obtain the weighted grayscale value of each pixel, and use the weighted grayscale value of each pixel in the foreground area to construct a histogram. Finally, equalize the histogram to obtain the enhanced image U1 of the foreground area.

[0043] Step S4: Improved limited contrast histogram equalization is performed on the background area to enhance the background area and limit noise, and finally an enhanced background area U2 is obtained. Aiming at the problem that the limited contrast histogram equalization method adopts a uniform clipping amplitude, which causes some sub-blocks to not be enhanced, the present invention determines the clipping amplitude of each sub-block according to the pixel grayscale value of each sub-block and clips and distributes the histogram of each sub-block to better improve the image quality.

[0044] Specifically, the background area is divided into n*n sub-blocks of equal size and non-overlapping, and the histogram of each sub-block is calculated. Then, the adaptive clipping amplitude T is set according to the uniformity of each sub-block. A lower clipping amplitude is set for uniform sub-blocks, and a higher clipping amplitude is set for uneven sub-blocks. The histogram is clipped according to the clipping amplitude, and the pixels are redistributed. The redistributed sub-block histogram is equalized, and finally the new pixel value is obtained by interpolation.

[0045] The uniformity β is expressed based on the average gray value Avg and the standard deviation σ.

[0046] β=σ / Avg

[0047] According to the uniformity formula, the grayscale average and standard deviation of the entire background area are calculated, and the grayscale average and standard deviation of each sub-block are calculated.

[0048] If the uniformity of the sub-block is less than that of the entire background area, then according to the pixel distribution of the sub-block histogram, the gray level of the current 10% of the pixels is selected, and the corresponding pixel numbers are pix1, pix2, ..., pix g , g is the grayscale level of the first 10%, and the clipping amplitude is obtained by the following formula.

[0049]

[0050] If the uniformity of the sub-block is greater than that of the entire background area, then according to the pixel distribution of the sub-block histogram, the gray level of the current 20% of the pixels is selected, and the corresponding pixel numbers are pix1, pix2, ..., pix h , h is the number of gray levels in the first 20%, and the clipping amplitude is obtained by the following formula.

[0051]

[0052] By setting the cropping amplitude by the above method, each sub-block can have a better image enhancement effect.

[0053] After the clipping amplitude is obtained, the histogram is clipped according to the clipping amplitude. In the process of redistributing the pixels, the total number of pixels exceeding the clipping amplitude is evenly distributed to each gray level that is not 0. The schematic diagram is as follows Figure 2 shown.

[0054] Step S5: Weighted superposition of the foreground area and the background area to obtain a fused image. Since the foreground and background use different histogram equalization methods, the contour of the area seam is too strong during fusion, so the overall grayscale of the enhanced area is adjusted. The formula is:

[0055]

[0056] In the formula, is the foreground area after grayscale adjustment, is the background area after grayscale adjustment, and M is the mean.

[0057] Finally, the two parts are fused to obtain the fused image U, whose formula is:

[0058]

[0059] Step S6: bilateral filtering is performed on the fused image to reduce noise. Since infrared images are affected by noise during acquisition and transmission, and after being enhanced by improved histogram equalization, some noise still affects the image effect, the present invention uses bilateral filtering to reduce noise in the image to better suppress noise.

[0060] Step S7: Get the final enhanced image and output it. Figure 3 as shown in (a) and (b).

[0061] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An infrared image enhancement method combining histogram equalization and bilateral filtering, characterized in that: The following steps are involved: Step 1: Collect infrared video and obtain original infrared image; Step 2: Segment the original infrared image to obtain the foreground area and the background area; Step 3: Perform histogram equalization of weighted grayscale values ​​on the foreground area; Step 4: Perform restricted contrast histogram equalization on the background area; Step 5: Superimpose the foreground area and the background area to obtain a fused image; Step 6: Perform bilateral filtering to reduce noise on the fused image; Step 7: Get the final enhanced image and output it.

2. The infrared image enhancement method according to claim 1, characterized in that: In step 2, threshold segmentation is used to roughly separate the foreground and background, and then a clustering algorithm is used to perform a finer segmentation. Finally, morphological operations are used to remove noise and fill the target area to make the area contour more complete; thus, the foreground area and background area of ​​the original infrared image are obtained.

3. The infrared image enhancement method according to claim 1 or 2, characterized in that: In step three, the local variance of the grayscale of the pixels in the foreground area is calculated one by one, and the weight of the grayscale of the pixel is obtained according to the local variance of the grayscale of the pixel. The grayscale value of each pixel is multiplied by the weight to obtain the weighted grayscale value of each pixel. The weighted grayscale value of each pixel in the foreground area is used to construct a histogram, and finally the histogram is equalized to obtain an enhanced image of the foreground area.

4. The infrared image enhancement method according to claim 3, characterized in that: In step 4, the background area is divided into n*n sub-blocks of equal size and non-overlapping, the histogram of each sub-block is calculated, and the adaptive clipping amplitude T is set according to the uniformity of each sub-block. A lower clipping amplitude is set for uniform sub-blocks, and a higher clipping amplitude is set for uneven sub-blocks; the histogram is clipped according to the clipping amplitude, and the pixels are redistributed; the redistributed sub-block histogram is equalized, and finally the new pixel value is obtained by interpolation operation; The uniformity β is expressed by the grayscale average value Avg and the grayscale standard deviation σ; β=σ / Avg If the uniformity of the sub-block is less than that of the entire background area, then according to the pixel distribution of the sub-block histogram, the gray level with the current 10% of the pixels is selected, and the corresponding pixel numbers are pix1, pix2, ..., pix g , g is the number of gray levels in the first 10%, and the clipping amplitude is obtained by the following formula; If the uniformity of the sub-block is greater than that of the entire background area, then according to the pixel distribution of the sub-block histogram, the gray level of the current 20% of the pixels is selected, and the corresponding pixel numbers are pix1, pix2, ..., pix h , h is the number of gray levels in the first 20%, and the clipping amplitude is obtained by the following formula; After the clipping amplitude is obtained, the histogram is clipped according to the clipping amplitude. In the process of redistributing the pixels, the total number of pixels exceeding the clipping amplitude is evenly distributed to each gray level that is not zero.

5. The infrared image enhancement method according to claim 1, characterized in that: In step five, since different histogram equalization methods are used for the foreground area and the background area, the contour at the area seam is too strong during fusion. The grayscale of the foreground area and the background area is adjusted before superimposition.

Images