Infrared image enhancement method and device, electronic equipment and storage medium

Abstract

The invention provides an infrared image enhancement method and device, electronic equipment and a storage medium, and relates to the technical field of image processing, and the method comprises the steps: determining a background gray level set and a foreground gray level set corresponding to an original infrared image; calculating cumulative probability distribution of each pixel in the original infrared image in each gray level in the background gray level set through a first cumulative distribution function to obtain a background cumulative histogram; calculating cumulative probability distribution of each pixel in the original infrared image in each gray level in the foreground gray level set through a second cumulative distribution function to obtain a foreground cumulative histogram; performing gray scale expansion on each gray scale in the original infrared image according to the background cumulative histogram and the foreground cumulative histogram, and establishing a gray scale mapping relationship between the expanded target gray scale and each gray scale in the original infrared image; and performing gray level conversion on each pixel in the original infrared image according to the gray level mapping relation to obtain a first enhanced infrared image, thereby optimizing the background enhancement of the infrared image.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an infrared image enhancement method, apparatus, electronic device, and storage medium. Background Technology

[0002] Compared to general visual images, infrared images have lower contrast, higher noise, and a narrower grayscale range, making subsequent detection and tracking more difficult. Histogram equalization is a commonly used image enhancement technique. Based on histogram equalization, improved image processing methods such as histogram plateau equalization and dual-plateau histogram equalization have been proposed. While these methods can improve the image quality of infrared images to some extent, their enhancement effect on specific targets is not ideal for certain fields, such as target recognition in autonomous driving. Summary of the Invention

[0003] This application provides an infrared image enhancement method, apparatus, electronic device, and storage medium to solve the technical problem of poor infrared image enhancement effect in the prior art.

[0004] In a first aspect, embodiments of this application provide an infrared image enhancement method, comprising: determining a background gray level set and a foreground gray level set corresponding to an original infrared image, wherein the background gray level set and the foreground gray level set each contain at least one gray level, and the value of any gray level in the foreground gray level set is greater than the value of the gray level in the background gray level set;

[0005] The cumulative probability distribution of each pixel in the original infrared image in the background gray level set is calculated by the first cumulative distribution function to obtain the background cumulative histogram.

[0006] The cumulative probability distribution of each pixel in the original infrared image in the foreground gray level set is calculated by the second cumulative distribution function to obtain the foreground cumulative histogram. The second cumulative distribution function is different from the first cumulative distribution function.

[0007] Based on the background cumulative histogram and the foreground cumulative histogram, gray levels in the original infrared image are expanded to establish a gray level mapping relationship between the expanded target gray level and the gray levels in the original infrared image.

[0008] The first enhanced infrared image is obtained by performing grayscale conversion on each pixel in the original infrared image according to the grayscale mapping relationship.

[0009] In one possible implementation, the first cumulative distribution function is:

[0010]

[0011] Where h1 represents the value of the maximum gray level in the background gray level set, T1(k) represents the cumulative probability of each pixel in the original infrared image being below gray level k, and p r (j) represents the proportion of pixels with gray level j to the total number of pixels.

[0012] In one possible implementation, the second cumulative distribution function is:

[0013]

[0014] Where h2 represents the value of the smallest gray level in the foreground gray level set, L represents the total number of gray levels corresponding to the image type of the original infrared image, and T2(k) represents the cumulative probability of each pixel in the original infrared image being below gray level k.

[0015] In one possible implementation, gray-level expansion is performed on each gray level in the original infrared image based on the background cumulative histogram and the foreground cumulative histogram, establishing a gray-level mapping relationship between the expanded target gray levels and the gray values ​​in the original infrared image, including:

[0016] Based on the background cumulative histogram and the foreground cumulative histogram, the cumulative probability of each gray level is normalized to generate a mixed cumulative histogram corresponding to the original infrared image. Based on the cumulative probability distribution of each gray level in the mixed cumulative histogram, the expanded target gray level corresponding to each gray level is calculated. The gray level mapping relationship between the target gray level and each gray value in the original infrared image is established.

[0017] In one possible implementation, determining the background grayscale set and the foreground grayscale set corresponding to the original infrared image includes: performing multiple iterations based on an initial grayscale threshold and the grayscale levels of each pixel in the original infrared image to determine a target grayscale threshold; constructing a background grayscale set using grayscale levels in the original infrared image that are not greater than the target grayscale threshold; and constructing a foreground grayscale set using grayscale levels in the original infrared image that are greater than the target grayscale threshold.

[0018] In one possible implementation, a target grayscale threshold is determined by performing multiple iterations based on an initial grayscale threshold and the grayscale levels of each pixel in the original infrared image. This includes: for the i-th iteration, dividing the original infrared image into a background pixel region and a foreground pixel region using the grayscale threshold corresponding to the i-th iteration; calculating the background-weighted average grayscale value of the background pixel region based on the proportion of background pixels at each grayscale level in the total background pixels; calculating the foreground-weighted average grayscale value of the foreground pixel region based on the proportion of foreground pixels at each grayscale level in the total foreground pixels; calculating the average of the background-weighted average grayscale value and the foreground-weighted average grayscale value as the grayscale mean of the i-th iteration; calculating the difference between the grayscale mean of the i-th iteration and the grayscale threshold corresponding to the i-th iteration; in response to the difference not meeting a preset condition, using the grayscale mean of the i-th iteration as the grayscale threshold corresponding to the (i+1)-th iteration to perform the (i+1)-th iteration; and in response to the difference meeting a preset condition, using the grayscale threshold corresponding to the i-th iteration as the target grayscale threshold.

[0019] In one possible implementation, after converting the grayscale values ​​of each pixel in the original infrared image according to the grayscale mapping relationship to obtain the first enhanced infrared image, the method further includes: performing a 3×3 neighborhood convolution operation on each pixel in the first enhanced infrared image based on a preset operator template to obtain the enhanced grayscale value corresponding to each pixel; and converting the grayscale value of each pixel in the first enhanced infrared image into the corresponding enhanced grayscale value to obtain the second enhanced infrared image.

[0020] Secondly, this application provides an infrared image enhancement device, including: a gray level division module, used to determine a background gray level set and a foreground gray level set corresponding to the original infrared image, wherein the background gray level set and the foreground gray level set each contain at least one gray level, and the value of any gray level in the foreground gray level set is greater than the value of the gray level in the background gray level set.

[0021] The histogram construction module is used to calculate the cumulative probability distribution of each pixel in the original infrared image in the background gray level set through the first cumulative distribution function, so as to obtain the background cumulative histogram.

[0022] The histogram construction module is also used to calculate the cumulative probability distribution of each pixel in the original infrared image in the foreground gray level set through the second cumulative distribution function, so as to obtain the foreground cumulative histogram. The second cumulative distribution function is different from the first cumulative distribution function.

[0023] The extension module is used to extend the gray levels of each gray level in the original infrared image based on the background cumulative histogram and the foreground cumulative histogram, and to establish the gray level mapping relationship between the extended target gray level and each gray level in the original infrared image.

[0024] The image enhancement module is used to perform grayscale conversion on each pixel in the original infrared image according to the grayscale mapping relationship to obtain the first enhanced infrared image.

[0025] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods of embodiments of this application when executing the computer program.

[0026] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of any one of the embodiments of this application.

[0027] Based on the method of this application embodiment, in addition to using histogram equalization to enhance infrared images, different cumulative distribution functions are used to construct foreground cumulative histograms and background cumulative histograms respectively, thereby realizing equalization enhancement of infrared images through different image enhancement strategies.

[0028] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0029] The accompanying drawings are provided for a better understanding of this solution and do not constitute a limitation of this application. Wherein:

[0030] Figure 1 This is a flowchart illustrating an exemplary embodiment of the infrared image enhancement method provided in this application. Figure 1 ;

[0031] Figure 2 This is a flowchart illustrating an exemplary embodiment of the infrared image enhancement method provided in this application. Figure 2 ;

[0032] Figure 3 This is a schematic diagram of the infrared image enhancement processing flow provided in an exemplary embodiment of this application;

[0033] Figure 4 This is a schematic diagram of an infrared image enhancement device provided in an exemplary embodiment of this application;

[0034] Figure 5 This is a block diagram of an electronic device used to implement the embodiments of this application. Detailed Implementation

[0035] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0036] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0037] Application scenarios

[0038] Histogram equalization is a technique that enhances image contrast by adjusting the grayscale distribution of image pixels. The core idea is to expand the relatively concentrated grayscale range in the image to a wider range, making the details of the image clearer.

[0039] The typical process for image enhancement using histogram equalization includes:

[0040] (1) Statistical analysis of the gray-level histogram of the original image.

[0041] This step can use histograms to count the number or frequency of pixels at each gray level (e.g., 0-255) in the original image, reflecting the brightness distribution of the image. If the image is dark, the histogram may be concentrated in the low gray value area; if the contrast is low, the histogram may be concentrated in a narrow gray range.

[0042] (2) Calculate the cumulative distribution function (CDF).

[0043] By using formula (1), the frequency of each gray level is accumulated successively to obtain the cumulative frequency from the smallest gray level to the current gray level.

[0044]

[0045] Where L represents the total number of gray levels corresponding to the image type of the original image, which is 256 in this case, p r (j) represents the proportion of pixels with gray level j to the total number of pixels, and T0(k) represents the cumulative probability of each pixel in the original infrared image being below gray level k, that is, the cumulative frequency from the minimum gray level to gray level k.

[0046] (3) Gray-scale mapping

[0047] Based on the cumulative distribution function, the original gray level is mapped to a new gray level using formula (2).

[0048] S k =int((L-1)×T0(k)) Formula (2)

[0049] Among them, S k Let be the gray level distribution function, which means that for any gray level in the original image, it is mapped to a new gray level by multiplying its corresponding cumulative frequency and the maximum gray level corresponding to the image type, and then rounding down.

[0050] (4) Generate the equalized enhanced image

[0051] Using the new gray levels of each pixel generated by formula (2), the gray values ​​of each pixel in the original image are adjusted. The gray values ​​that appear frequently in the pixels of the original image are expanded to a wider gray value range, increasing the gray value difference between pixels and improving the overall contrast of the image.

[0052] Based on this, platform histogram equalization is an improved method of histogram equalization. By selecting an appropriate platform threshold, it is corrected during the histogram equalization process. If the histogram statistics of a certain gray level are greater than the platform threshold, then the histogram statistics are set to the platform threshold; if the histogram statistics of a certain gray level are less than the platform threshold, then the histogram statistics are kept unchanged, thereby constraining the background noise.

[0053] Dual-platform histogram equalization is a further improvement on platform histogram equalization. It introduces two platform thresholds (an upper threshold and a lower threshold) to correct the histogram equalization process. If the histogram statistic of a grayscale level is greater than the upper platform threshold, then that histogram statistic is set to the upper platform threshold. If the histogram statistic of a grayscale level is less than the lower platform threshold but greater than 0, then that histogram statistic is set to the lower platform threshold. If the histogram statistic of a grayscale level is between the upper and lower thresholds, then that histogram statistic remains unchanged. This approach uses the upper threshold to constrain background noise while using the lower threshold to amplify subtle target details, thus preserving low-level grayscale detail features during histogram equalization.

[0054] However, whether it is traditional histogram equalization or improved platform histogram equalization and dual-platform histogram equalization, the same enhancement strategy is used for the background and foreground within the platform threshold, which is difficult to adapt to the diverse needs of different scenarios.

[0055] For example, in the field of autonomous driving, when vehicles perceive objects in the environment using infrared images, sometimes for foreground parts such as heated objects, it is not necessary to pay too much attention to the details of the heated objects; only the outline of the heated objects is needed. However, for background objects, such as lane lines and green belts, it is necessary to know their specific locations. Existing infrared image enhancement methods using histogram equalization, which equalize and enhance based on the frequency of occurrence of different gray levels, often only improve the detailed features of the foreground image region. They cannot make non-heated objects (such as lane lines and green belts) in the background region (such as lane lines and green belts) more clearly distinguishable from the surrounding environment, and cannot help the target detection algorithms applied to autonomous driving to quickly and accurately identify objects in the background image region.

[0056] Based on this, this application proposes an infrared image enhancement method that can construct foreground cumulative histograms and background cumulative histograms using different cumulative distribution functions. On this basis, histogram equalization is performed on the background and foreground regions of the infrared image, thereby enabling image enhancement using different enhancement strategies for the background and foreground regions and improving the scene applicability of histogram equalization.

[0057] Exemplary methods

[0058] Figure 1 This is a schematic flowchart of an exemplary embodiment of the infrared image enhancement method provided in this application. This embodiment can be applied to electronic devices, such as... Figure 1 As shown, the method includes the following steps:

[0059] Step S101: Determine the background gray level set and the foreground gray level set corresponding to the original infrared image. The background gray level set and the foreground gray level set each contain at least one gray level, and the value of any gray level in the foreground gray level set is greater than the value of the gray level in the background gray level set.

[0060] Here, due to the characteristics of infrared imaging, the foreground and background exhibit significant grayscale differences in infrared images due to substantial temperature differences. The foreground and background in the infrared image can be separated by setting a grayscale threshold. After separating the background and foreground, a background grayscale set and a foreground grayscale set can be constructed based on the grayscale levels corresponding to the pixels in the background and foreground parts, respectively.

[0061] For example, in step S101, determining the background grayscale set and the foreground grayscale set corresponding to the original infrared image includes: performing multiple iterations based on an initial grayscale threshold and the grayscale levels of each pixel in the original infrared image to determine a target grayscale threshold; constructing a background grayscale set using grayscale levels in the original infrared image that are not greater than the target grayscale threshold; and constructing a foreground grayscale set using grayscale levels in the original infrared image that are greater than the target grayscale threshold.

[0062] For an 8-bit infrared image, which includes 256 gray levels from 0 to 255, the initial gray level threshold can be set to a random value between 0 and 255. Specifically, the iterative process of determining the target gray level threshold by performing multiple rounds of iteration based on the initial gray level threshold and the gray levels of each pixel in the original infrared image is as follows:

[0063] 1) For the i-th round of multiple iterations, the original infrared image is divided into background pixel region and foreground pixel region using the grayscale threshold corresponding to the i-th round of iteration.

[0064] In the initial stage, pixels with a gray level lower than the initial gray level threshold are considered as background pixels, and pixels with a gray level higher than the initial gray level threshold are considered as foreground pixels, thus dividing the initial background pixel region and the initial foreground pixel region.

[0065] 2) Based on the proportion of background pixels of each gray level in the total background pixels, calculate the weighted average gray value of the background pixel region. The formula for calculating the weighted average gray value is as follows:

[0066] μ B =∑ (a,b)∈object f(a,b) / N B Formula (3)

[0067] Where, N B The number of pixels in the background pixel region is represented by (a, b), which represent the horizontal and vertical coordinates of the current pixel in the original infrared image, respectively.

[0068] 3) Based on the proportion of foreground pixels at each gray level in the total foreground pixels, calculate the foreground weighted average gray value of the foreground pixel region. The formula for calculating the background weighted average gray value is as follows:

[0069] μ o =∑ (a,b)∈object f(a,b) / N o Formula (4)

[0070] Where, N o This indicates the number of pixels in the foreground pixel region.

[0071] 4) Calculate the average of the background weighted average gray value and the foreground weighted average gray value as the gray average value of the i-th iteration. Calculate the difference between the gray average value of the i-th iteration and the gray threshold corresponding to the i-th iteration. Determine whether to start a new round of iteration or stop iteration based on whether the difference meets the preset conditions.

[0072] Th(i+1)=(μ B (i)+μ o Formula (5) (i)) / 2

[0073] Where, μ B (i) and μ o (i) represents the background weighted average gray value and the foreground weighted average gray value of the i-th iteration, and Th(i+1) represents the gray threshold of the (i+1)-th iteration.

[0074] In other words, whether the difference meets the preset conditions has the following two possibilities:

[0075] Case 1: In response to the difference not meeting the preset condition, the gray average value of the i-th iteration is used as the gray threshold corresponding to the i+1-th iteration to execute the i+1-th iteration.

[0076] Case 2: In response to the difference meeting the preset conditions, the grayscale threshold corresponding to the i-th iteration is used as the target grayscale threshold.

[0077] In one possible implementation, the preset condition can be that the difference is less than a certain specific value, such as 10, 20, 25, etc. Since the gray threshold of the i-th iteration is also the gray mean of the (i-1)-th iteration, the difference between the gray mean of the i-th iteration and the gray threshold corresponding to the i-th iteration can be regarded as the difference between the gray mean of two consecutive iterations. When the difference between the gray mean of two consecutive iterations is less than a specific value, it reflects that if the iteration continues, the average gray difference between the pixels in the foreground and background regions will be smaller. That is, the foreground and background regions in the current iteration have already divided the pixels with certain gray differences in the original image into foreground and background, and the iteration can be terminated, with the gray threshold of the current iteration as the target gray threshold.

[0078] The method in this embodiment can calculate the foreground-background distinction threshold applicable to the original infrared image based on an iterative algorithm. Compared with using a fixed distinction threshold to divide the foreground and background, it can improve the segmentation accuracy and facilitate subsequent image enhancement using different enhancement strategies for the foreground and background regions.

[0079] It should be noted that the foreground and background division method shown in this embodiment is only an example of determining the background and foreground regions, serving as a basis for subsequent image enhancement of the background and foreground regions using different enhancement strategies. It is not intended to limit the implementation of the infrared image enhancement method in this application. In practical applications, other means of dividing the foreground and background regions can be adaptively selected and combined with the infrared image enhancement method of this application for image enhancement.

[0080] Step S102: Calculate the cumulative probability distribution of each pixel in the original infrared image in the background gray level set using the first cumulative distribution function to obtain the background cumulative histogram.

[0081] Step S103: Calculate the cumulative probability distribution of each pixel in the original infrared image at each gray level in the foreground gray level set using the second cumulative distribution function to obtain the foreground cumulative histogram. The second cumulative distribution function is different from the first cumulative distribution function.

[0082] For example, the first cumulative distribution function is:

[0083]

[0084] Where h1 represents the value of the maximum gray level in the background gray level set, T1(k) represents the cumulative probability of each pixel in the original infrared image being below gray level k, and p r (j) represents the proportion of pixels with gray level j to the total number of pixels.

[0085] The second cumulative distribution function is:

[0086]

[0087] Where h2 represents the value of the smallest gray level in the foreground gray level set, L represents the total number of gray levels corresponding to the image type of the original infrared image, and T2(k) represents the cumulative probability of each pixel in the original infrared image being below gray level k.

[0088] For example, if the original infrared image is an 8-bit image (L = 256), the total number of pixels in the original infrared image is n = 1000, a certain gray level in the background region is k1 = 100, and the number of pixels corresponding to k1 is n1 = 200, then p r (100)=0.2, log2(100+1)≈6.66, the contribution of the background pixel with gray level 100 to the background accumulation is 0.2×6.66=1.332. If a certain gray level k2=200 in the foreground region, and the number of pixels corresponding to k2 is n2=200, then p r If (200) = 0.2, then the contribution of the foreground pixel with gray level 200 to the foreground accumulation is 0.2.

[0089] It is understandable that, compared to the cumulative distribution function of conventional histogram equalization in formula (1), the first cumulative distribution function used to generate the background cumulative histogram introduces log2(j+1) when calculating the cumulative probability of each gray level in the background gray level set. Since gray level j is a positive integer not less than 1, the result of log2(j+1) is a value greater than or equal to 1. Thus, the first cumulative distribution function in this embodiment is essentially an improvement on the existing cumulative distribution function. In the process of calculating the cumulative probability, the background pixels of each gray level are additionally weighted, so that the cumulative probability of each gray level in the generated background cumulative histogram is higher than the actual probability, so that in the subsequent histogram equalization process, the gray level mapping transformation of the background pixels can be higher and the contrast stronger.

[0090] In this embodiment, when the second cumulative distribution function calculates the cumulative probability of each gray level in the foreground gray level set, it uses the calculation method in the first cumulative distribution function to calculate the cumulative probability of the background gray level while maintaining the linear accumulation consistent with formula (1). This allows the cumulative probability calculation result of the first cumulative distribution function to adjust the cumulative probability distribution of each gray level in the foreground gray level set, avoiding the weighting of the cumulative probability of the background gray level in the first cumulative distribution function, which would result in insufficient enhancement of the foreground pixels during the final image equalization. This allows for additional enhancement of the background object while ensuring the overall image enhancement effect.

[0091] Step S104: Based on the background cumulative histogram and the foreground cumulative histogram, perform gray-level expansion on each gray level in the original infrared image to establish a gray-level mapping relationship between the expanded target gray level and each gray level in the original infrared image.

[0092] Here, since the cumulative probability of each gray level in the background gray level set is weighted and enhanced when generating the background cumulative histogram, the total distribution probability of each gray level obtained by combining the background cumulative histogram and the foreground cumulative histogram will eventually be greater than 1. Therefore, before performing equalization image enhancement using the background cumulative histogram and the foreground cumulative histogram in this embodiment, step S104 further includes:

[0093] Based on the background cumulative histogram and the foreground cumulative histogram, the cumulative probability of each gray level is normalized to generate a mixed cumulative histogram corresponding to the original infrared image. Based on the cumulative probability distribution of each gray level in the mixed cumulative histogram, the expanded target gray level corresponding to each gray level is calculated. The gray level mapping relationship between the target gray level and each gray value in the original infrared image is established.

[0094]

[0095] Where T3(k) is the normalized cumulative probability distribution corresponding to gray level k, that is, the cumulative probability distribution of each gray level in the mixed cumulative histogram, T hb (k) is the mixed cumulative distribution function of the first cumulative distribution function T1(k) and the second cumulative distribution function T2(k), where T is the cumulative distribution function of the gray level k when it belongs to the background gray level set. hb (k) is T1(k), when gray level k belongs to the foreground gray level set. hb (k) is T2(k).

[0096] Based on the cumulative probability distribution of each gray level in the mixed cumulative histogram, calculate the expanded target gray level corresponding to each gray level.

[0097] S k =int((L-1)×T3(k)) Formula (9)

[0098] Among them, S k ' represents the expanded gray level distribution function. Based on formula (9), the expanded target gray level corresponding to each gray level in the original image can be calculated.

[0099] Step S105: Perform grayscale conversion on each pixel in the original infrared image according to the grayscale mapping relationship to obtain the first enhanced infrared image.

[0100] For example, if a certain original gray level in the original infrared image is 50, the S obtained after calculation using formula (9) k If '=75, then during histogram equalization, the gray level of all pixels with a gray level of 50 will be adjusted to 75.

[0101] The method in this embodiment enhances infrared images by using histogram equalization, and constructs foreground and background cumulative histograms using different cumulative distribution functions, thereby achieving equalization enhancement of infrared images through different image enhancement strategies.

[0102] Specifically, in this embodiment, the probability distribution of the background gray level is weighted and enhanced in the first cumulative distribution function corresponding to the construction of the background cumulative histogram. This can make the gray level of the background pixel gray level mapping transformation higher and the contrast stronger, and can provide additional enhancement to the background object while ensuring the overall image enhancement effect.

[0103] For example, such as Figure 2 As shown, after converting the grayscale values ​​of each pixel in the original infrared image according to the grayscale mapping relationship in step S105 to obtain the first enhanced infrared image, the method further includes:

[0104] Step S106: Perform a 3×3 neighborhood convolution operation on each pixel in the first enhanced infrared image based on the preset operator template to obtain the enhanced grayscale value corresponding to each pixel;

[0105] Step S107: Convert the grayscale value of each pixel in the first enhanced infrared image into the corresponding enhanced grayscale value to obtain the second enhanced infrared image.

[0106] It is understandable that, since the edges where the foreground and background meet in infrared images are not as obvious as in conventional visual images, in steps S101 to S105 of this application, during the process of histogram equalization enhancement of the infrared image, the gray values ​​of the background objects are enhanced more. This may cause the gray values ​​of the edges of the foreground pixel area to be closer to those of the background pixel area, thus affecting the accurate recognition of the foreground contour. Therefore, after obtaining the first enhanced infrared image in step S105, edge enhancement is still required for the first enhanced infrared image.

[0107] Specifically, the edge enhancement of the first enhanced infrared image can be performed using the following formula:

[0108]

[0109] Where F represents the second enhanced infrared image after edge enhancement, I represents the first enhanced infrared image, and H is a preset Laplace operator template with a center pixel weight of 5 to enhance brightness preservation and a surrounding 5-neighbor pixel weight of -1 to highlight edge differences.

[0110] For example, suppose a pixel has a gray value of 100 and the gray values ​​of its neighboring pixels are all 80. Then the enhanced gray value of this pixel is S1 = 5 × 100 - 80 × 4 = 180, which improves the contrast between this pixel and its neighboring pixels and achieves edge enhancement. If the gray values ​​of the neighboring pixels are all 100, then the enhanced gray value is S2 = 5 × 100 - 100 × 4 = 100. The surface of this pixel remains unchanged for areas with uniform gray value distribution after edge enhancement.

[0111] In this embodiment, the high-frequency components (regions with rapidly changing gray values) in the first enhanced infrared image after histogram equalization are actually enhanced in a targeted manner, which significantly improves the recognizability of the target contour and makes up for the lack of detail enhancement by histogram equalization.

[0112] Figure 3 This is a schematic diagram of an infrared image enhancement processing flow provided for an exemplary embodiment of this application.

[0113] like Figure 3As shown, after inputting the original infrared image, the computer can determine a suitable foreground and background segmentation threshold through iterative iteration based on the grayscale distribution of the input image. Based on the foreground and background segmentation threshold, the computer can distinguish the foreground grayscale set and the background grayscale set. Based on the first cumulative distribution function, the computer can calculate the cumulative probability distribution of each pixel in the original infrared image in the background grayscale set to obtain the background cumulative histogram. Based on the second cumulative distribution function, the computer can calculate the cumulative probability distribution of each pixel in the original infrared image in the foreground grayscale set to obtain the foreground cumulative histogram.

[0114] Furthermore, after obtaining the background cumulative histogram and the foreground cumulative histogram, the cumulative probabilities of each gray level can be normalized by combining the background cumulative histogram and the foreground cumulative histogram to generate the mixed cumulative histogram corresponding to the original infrared image. Based on the cumulative probability distribution of each gray level in the mixed cumulative histogram, gray level expansion can be performed on each gray level in the original infrared image. Based on the expanded gray levels, gray level conversion can be performed on the gray levels of each pixel in the original infrared image through a mapping method to obtain the first enhanced infrared image.

[0115] After obtaining the first enhanced infrared image, in order to avoid the gray levels at the junction of the foreground and background regions being too similar due to gray level conversion, resulting in indistinct edges and affecting the contour recognition of the foreground object, the high-frequency components in the first enhanced infrared image can be enhanced to enhance the edge information of the foreground region, improve the gray level difference between the foreground and background regions at the junction, and improve the recognizability of the target contour, thus obtaining the second enhanced infrared image, which supplements the shortcomings of histogram equalization in enhancing details.

[0116] The specific settings and implementation methods of the embodiments of this application have been described above from different perspectives. Using the methods provided in the above embodiments, based on infrared image enhancement using histogram equalization, foreground cumulative histograms and background cumulative histograms are constructed using different cumulative distribution functions, thereby achieving equalization enhancement of infrared images through different image enhancement strategies.

[0117] It should also be emphasized that the first cumulative distribution function used to construct the background cumulative histogram in this application is essentially an improvement on the existing cumulative distribution function. In the process of calculating the cumulative probability, each gray level of the background pixel is additionally weighted, so that the cumulative probability of each gray level in the generated background cumulative histogram is higher than the actual probability. This allows the gray level mapping transformation of the background pixels to be higher and the contrast to be stronger in the subsequent histogram equalization process.

[0118] Exemplary device

[0119] As an implementation of the above methods, such as Figure 4 As shown in the figure, this application embodiment also provides an infrared image enhancement device, which may include:

[0120] The gray level segmentation module 401 is used to determine the background gray level set and the foreground gray level set corresponding to the original infrared image. The background gray level set and the foreground gray level set each contain at least one gray level, and the value of any gray level in the foreground gray level set is greater than the value of the gray level in the background gray level set.

[0121] The histogram construction module 402 is used to calculate the cumulative probability distribution of each pixel in the original infrared image in the background gray level set through the first cumulative distribution function, so as to obtain the background cumulative histogram.

[0122] The histogram construction module 402 is also used to calculate the cumulative probability distribution of each pixel in the original infrared image in the foreground gray level set through the second cumulative distribution function, so as to obtain the foreground cumulative histogram, wherein the second cumulative distribution function is different from the first cumulative distribution function;

[0123] The extension module 403 is used to extend the gray levels of each gray level in the original infrared image according to the background cumulative histogram and the foreground cumulative histogram, and to establish the gray level mapping relationship between the extended target gray level and each gray level in the original infrared image.

[0124] The image enhancement module 404 is used to perform grayscale conversion on each pixel in the original infrared image according to the grayscale mapping relationship to obtain the first enhanced infrared image.

[0125] For example, the first cumulative distribution function is:

[0126]

[0127] Where h1 represents the value of the maximum gray level in the background gray level set, T1(k) represents the cumulative probability of each pixel in the original infrared image being below gray level k, and p r (j) represents the proportion of pixels with gray level j to the total number of pixels.

[0128] For example, the second cumulative distribution function is:

[0129]

[0130] Where h2 represents the value of the smallest gray level in the foreground gray level set, L represents the total number of gray levels corresponding to the image type of the original infrared image, and T2(k) represents the cumulative probability of each pixel in the original infrared image being below gray level k.

[0131] For example, extension module 403 is used for:

[0132] Based on the background cumulative histogram and the foreground cumulative histogram, the cumulative probability of each gray level is normalized to generate a mixed cumulative histogram corresponding to the original infrared image. Based on the cumulative probability distribution of each gray level in the mixed cumulative histogram, the expanded target gray level corresponding to each gray level is calculated. The gray level mapping relationship between the target gray level and each gray value in the original infrared image is established.

[0133] For example, the grayscale segmentation module 401 is used for:

[0134] The target gray level is determined by multiple iterations based on the initial gray level threshold and the gray level of each pixel in the original infrared image. A background gray level set is constructed using the gray levels in the original infrared image that are not greater than the target gray level threshold. A foreground gray level set is constructed using the gray levels in the original infrared image that are greater than the target gray level threshold.

[0135] For example, the grayscale segmentation module 401 is also used for:

[0136] For the i-th iteration in a multi-round iteration, the original infrared image is divided into a background pixel region and a foreground pixel region using the grayscale threshold corresponding to the i-th iteration. The background-weighted average grayscale value of the background pixel region is calculated based on the proportion of background pixels of each grayscale level in the total background pixels. The foreground-weighted average grayscale value of the foreground pixel region is calculated based on the proportion of foreground pixels of each grayscale level in the total foreground pixels. The average of the background-weighted average grayscale value and the foreground-weighted average grayscale value is calculated as the grayscale mean of the i-th iteration. The difference between the grayscale mean of the i-th iteration and the grayscale threshold corresponding to the i-th iteration is calculated.

[0137] In response to the difference not meeting the preset condition, the gray mean of the i-th iteration is used as the gray threshold corresponding to the i+1-th iteration to execute the i+1-th iteration;

[0138] In response to the difference meeting the preset condition, the grayscale threshold corresponding to the i-th iteration is used as the target grayscale threshold.

[0139] For example, the image enhancement module 404 is further configured to: perform a 3×3 neighborhood convolution operation on each pixel in the first enhanced infrared image based on a preset operator template to obtain the enhanced grayscale value corresponding to each pixel; and convert the grayscale value of each pixel in the first enhanced infrared image into the corresponding enhanced grayscale value to obtain the second enhanced infrared image.

[0140] The functions of each unit, module, or sub-module in the various devices of this application embodiment can be found in the corresponding descriptions in the above method embodiments, and they have corresponding beneficial effects, which will not be repeated here.

[0141] Exemplary electronic devices and computer-readable storage media

[0142] Figure 5 This is a block diagram of an electronic device used to implement embodiments of this application. Figure 5 As shown, the electronic device includes a memory 501 and a processor 502. The memory 501 stores a computer program that can run on the processor 502. When the processor 502 executes the computer program, it implements the method described in the above embodiments. The number of memories 501 and processors 502 can be one or more.

[0143] The electronic device also includes:

[0144] Communication interface 503 is used to communicate with external devices and perform data exchange and transmission.

[0145] If the memory 501, processor 502, and communication interface 503 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0146] Optionally, in a specific implementation, if the memory 501, processor 502, and communication interface 503 are integrated on a single chip, then the memory 501, processor 502, and communication interface 503 can communicate with each other through an internal interface.

[0147] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0148] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in this application.

[0149] It should be understood that the aforementioned processor can be a CPU, or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), FPGAs, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0150] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0151] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0152] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions that, when executed by a processor, cause the processor to perform the steps of the image processing methods of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0153] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0154] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0155] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0156] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0157] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0158] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0159] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0160] The above description is merely an exemplary embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An infrared image enhancement method, characterized in that, include: The background gray level set and the foreground gray level set corresponding to the original infrared image are determined. The background gray level set and the foreground gray level set each contain at least one gray level, and the value of any gray level in the foreground gray level set is greater than the value of the gray level in the background gray level set. The cumulative probability distribution of each pixel in the original infrared image in the background gray level set is calculated by the first cumulative distribution function to obtain the background cumulative histogram. The cumulative probability distribution of each pixel in the original infrared image in the foreground gray level set is calculated by the second cumulative distribution function to obtain the foreground cumulative histogram. The second cumulative distribution function is different from the first cumulative distribution function. The gray levels in the original infrared image are expanded according to the background cumulative histogram and the foreground cumulative histogram, and a gray level mapping relationship is established between the expanded target gray level and the gray levels in the original infrared image. The first enhanced infrared image is obtained by performing grayscale conversion on each pixel in the original infrared image according to the grayscale mapping relationship.

2. The method according to claim 1, characterized in that, The first cumulative distribution function is: Where h1 represents the value of the maximum gray level in the background gray level set, T1(k) represents the cumulative probability of each pixel in the original infrared image being below gray level k, and p r (j) represents the proportion of pixels with gray level j to the total number of pixels.

3. The method according to claim 2, characterized in that, The second cumulative distribution function is: Where h2 represents the value of the smallest gray level in the foreground gray level set, L represents the total number of gray levels corresponding to the image type of the original infrared image, and T2(k) represents the cumulative probability of each pixel in the original infrared image being below gray level k.

4. The method according to claim 1, characterized in that, Based on the background cumulative histogram and the foreground cumulative histogram, gray-level expansion is performed on each gray level in the original infrared image to establish a gray-level mapping relationship between the expanded target gray levels and each gray value in the original infrared image, including: Based on the background cumulative histogram and the foreground cumulative histogram, the cumulative probability of each gray level is normalized to generate the mixed cumulative histogram corresponding to the original infrared image. Based on the cumulative probability distribution of each gray level in the hybrid cumulative histogram, the expanded target gray level corresponding to each gray level is calculated. Establish a gray-level mapping relationship between the target gray level and each gray value in the original infrared image.

5. The method according to claim 1, characterized in that, The background grayscale set and foreground grayscale set corresponding to the original infrared image are determined, including: The target grayscale threshold is determined by multiple iterations based on the initial grayscale threshold and the grayscale level of each pixel in the original infrared image. The background gray level set is constructed using the gray levels in the original infrared image that are not greater than the target gray level threshold; The foreground gray level set is constructed using the gray levels in the original infrared image that are greater than the target gray level threshold.

6. The method according to claim 5, characterized in that, The target grayscale threshold is determined through multiple iterations based on an initial grayscale threshold and the grayscale levels of each pixel in the original infrared image, including: For the i-th round of the multi-round iteration, the original infrared image is divided into a background pixel region and a foreground pixel region using the grayscale threshold corresponding to the i-th round iteration. Calculate the background weighted average gray value of the background pixel region based on the proportion of each gray level of the background pixel region in the total background pixels; Calculate the foreground weighted average gray value of the foreground pixel region based on the proportion of foreground pixels of each gray level in the total foreground pixels; Calculate the average of the background weighted average gray value and the foreground weighted average gray value, and use it as the gray average value of the i-th iteration; Calculate the difference between the mean gray value of the i-th iteration and the gray value threshold corresponding to the i-th iteration; In response to the difference not meeting the preset condition, the average gray value of the i-th iteration is used as the gray value threshold corresponding to the (i+1)-th iteration to execute the (i+1)-th iteration; In response to the difference satisfying a preset condition, the grayscale threshold corresponding to the i-th iteration is taken as the target grayscale threshold.

7. The method according to any one of claims 1-6, characterized in that, After converting the grayscale values ​​of each pixel in the original infrared image according to the grayscale mapping relationship to obtain the first enhanced infrared image, the method further includes: Based on the preset operator template, a 3×3 neighborhood convolution operation is performed on each pixel in the first enhanced infrared image to obtain the enhanced gray value corresponding to each pixel; The grayscale value of each pixel in the first enhanced infrared image is converted into the corresponding enhanced grayscale value to obtain the second enhanced infrared image.

8. An infrared image enhancement device, characterized in that, include: The gray level segmentation module is used to determine the background gray level set and the foreground gray level set corresponding to the original infrared image. The background gray level set and the foreground gray level set each contain at least one gray level, and the value of any gray level in the foreground gray level set is greater than the value of the gray level in the background gray level set. The histogram construction module is used to calculate the cumulative probability distribution of each pixel in the original infrared image in the background gray level set through the first cumulative distribution function, so as to obtain the background cumulative histogram. The histogram construction module is further configured to calculate the cumulative probability distribution of each pixel in the original infrared image in the foreground gray level set using a second cumulative distribution function, thereby obtaining a foreground cumulative histogram, wherein the second cumulative distribution function is different from the first cumulative distribution function; The extension module is used to perform gray-level expansion on each gray level in the original infrared image according to the background cumulative histogram and the foreground cumulative histogram, and to establish a gray-level mapping relationship between the expanded target gray level and each gray level in the original infrared image. The image enhancement module is used to perform grayscale conversion on each pixel in the original infrared image according to the grayscale mapping relationship to obtain a first enhanced infrared image.

9. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-7.

Images