Thermal infrared image processing method for improving comfort level of human eyes
Through time domain and airspace filtering, linear mapping, adaptive histogram equalization and S-curve adjustment, combined with pseudo-color color plate mapping, the problem of poor visual effects caused by existing infrared image processing algorithms in noise amplification is solved, and the visual effect of existing infrared image processing algorithms is significantly improved. The contrast and brightness of the image enhances visual effects and comfort.
Patent Information
- Application Number
- CN202411091618.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-03
AI Technical Summary
The existing infrared image human eye comfort improvement algorithms are prone to amplification of noise when processing images with low contrast or large brightness distribution, resulting in poor visual effects and failing to effectively consider the time domain characteristics of image brightness distribution.
Using a combination of time domain filtering and airspace filtering, image noise is removed through time domain filtering, vertical stripes and dot noise are removed, and the contrast and brightness of the image is improved through linear mapping, adaptive histogram equalization and S-curve adjustment. , combined with pseudo-color color palette mapping, enhance image details and visual effects.
It significantly improves the visual effect of infrared images, enhances the contrast and brightness of the image, reduces noise interference, improves the visual effect of the image and the visual effect of the details, and improves the comfort of the image through pseudo-color color palette mapping.
Smart Images

Figure CN119071601A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to a thermal infrared image processing method for improving the comfort of human eyes. Background Art
[0002] Infrared images are an important type of multispectral perception data. They have observation capabilities that traditional visible light images do not have in complex scenes such as night, smoke, and fog. However, due to the characteristics of sensors and human vision, infrared images usually need to be processed to improve the comfort of the human eye in practical applications to obtain better visual effects. At present, the commonly used methods to improve the comfort of the human eye are mainly traditional spatial domain algorithms, which ignore the time domain features contained in the time domain continuity of the image. At the same time, with the improvement of unsupervised learning algorithms, algorithm tuning techniques using unsupervised methods are gradually showing their advantages in effect, but in the field of image processing algorithms, there are not many studies on related algorithm tuning techniques.
[0003] The current mainstream algorithms for improving the comfort of infrared images to the human eye include image truncation and stretching, histogram equalization, and brightness gamma transform.
[0004] Image truncation and stretching determines the high and low thresholds of brightness in the image and stretches the pixel values within the range to improve the visual effect of the image. This method can eliminate the interference of abnormal pixel values in the image and has a certain robustness, but for images with low contrast, this stretching method will amplify the noise in the image and affect the image perception. At the same time, this method does not consider the time domain characteristics of image brightness distribution, and the image brightness changes abruptly when the scene switches.
[0005] Histogram equalization is a commonly used image enhancement technique that improves contrast by converting the histogram distribution of an image into a uniform distribution. This method can significantly enhance the overall contrast of an image, but for some images with large differences in brightness distribution, it is difficult to significantly improve the contrast of locally brighter or darker areas. At the same time, this method may cause the overall contrast of the image to be over-enhanced, amplifying the noise in the image.
[0006] The brightness gamma transform method is a method of adjusting the brightness of an image using a gamma mapping curve. This mapping method can improve the brightness and contrast of darker areas of the image. However, this method also increases the brightness of brighter areas of the image, causing the contrast of these areas to decrease, affecting the visual effect. Summary of the invention
[0007] In view of the defects existing in the above-mentioned background technology, the present invention provides a thermal infrared image processing method for improving the comfort of human eyes. The thermal infrared image is processed by this method, which can significantly improve the comfort of human eyes.
[0008] The present invention is implemented by the following technical solutions:
[0009] A thermal infrared image processing method for improving human eye comfort, the method comprising the following steps:
[0010] 1) Original image filtering: filtering the original image to reduce the noise of the original image; the filtering includes time domain filtering and spatial domain filtering; the specific method of the time domain filtering is: for a single infrared image input, the current frame image is subjected to time domain filtering according to the previous frame image to remove part of the time domain noise in the image; the spatial domain filtering is used to remove the vertical stripes and point noise in the infrared image after the time domain filtering;
[0011] 2) Contrast / brightness adjustment: First, the filtered image of the original image is linearly mapped in the spatial domain, thereby converting the 14-bit original image into an 8-bit grayscale image; then, the grayscale image is adaptively histogram equalized with limited contrast to enhance the contrast of the image; finally, the brightness and contrast of the image are further adjusted using an S curve; this step can achieve two functions: first, in the spatial domain, the overall contrast / brightness of the thermal infrared image is adaptively linearly mapped to improve the overall quality of the image; second, in the temporal domain, the linear mapping range of the original image is controlled to reduce the sudden change in brightness of the image caused by scene changes and improve the image perception;
[0012] 3) Detail enhancement: performing neighborhood mean filtering on each pixel of the image after contrast / brightness adjustment to obtain the low-frequency part of the image, subtracting the extracted low-frequency part from the original image to generate the high-frequency details of the image, scaling the high-frequency details and superimposing them on the original image for image enhancement, thereby improving the visual effect of the detail part;
[0013] 4) Pseudo-color palette mapping: applying a pseudo-color palette to the image after detail enhancement to improve the viewing comfort of the image; the method for obtaining the pseudo-color palette is: uniformly selecting a number of pixel values in the range of 0-255, manually adjusting the mapped pixel values, using these points to perform cubic spline interpolation to generate a mapping curve of the pseudo-color palette, manually adjusting the mapped pixel values according to the mapping effect of the pseudo-color palette, and designing a pseudo-color palette.
[0014] In the above technical solution, further, the step 1) specifically includes the following steps:
[0015] (1) Temporal filtering: Temporal filtering is performed on the current frame image based on the previous frame image to remove some temporal noise in the image. The process can be expressed as
[0016] I tfilt (t)=(1-W(t))☉I raw (t)+W(t)☉I raw(t-1),#(1)
[0017] Among them, W(t) represents the weight matrix at time t, I tfilt (t) represents the image after time domain filtering at time t, I raw (t) and I raw (t-1) are the original image of the current frame and the original image of the previous frame respectively, ⊙ is the Hadamard product of the matrix;
[0018] The calculation process of the weight matrix W(t) can be expressed as
[0019] W(t)=α*exp(-|I raw (t)-I raw (t-1)| / h),#(2)
[0020] Among them, α and h are parameters. By adjusting the values of parameters α and h, the weights of the pixel values of the current frame and the previous frame in the weighted sum can be controlled. The greater the difference between adjacent frames, the smaller the weight given to the previous frame. In this way, while introducing the previous frame to eliminate noise, as much information as possible of the current frame is retained.
[0021] (2) Spatial filtering: including vertical stripe removal and non-local mean filtering, used to reduce spatial noise;
[0022] a. Remove vertical stripes: Since each detection element of the infrared detector has a different response rate to infrared radiation, the original image collected has obvious vertical stripes. Therefore, it is necessary to remove the vertical stripes from the original image after time domain filtering. The specific steps of removing vertical stripes are:
[0023] First, the image is subjected to a Gaussian filter with a horizontal window. The calculation process can be expressed as
[0024]
[0025] Among them, kernel represents the Gaussian kernel used in Gaussian filtering, R is the radius of the Gaussian kernel, and I tfilt (x+r,y) represents image I tfilt The pixel value of the coordinate point (x+r,y), I gfilt (x,y) represents the image I after horizontal window Gaussian filtering gfilt The pixel value at the midpoint (x, y).
[0026] Next, the pixel difference of each pixel in the image before and after Gaussian filtering is calculated. If the difference is less than the set threshold, it is determined that the pixel value of the pixel is similar to the horizontal neighborhood, and there is a vertical stripe phenomenon. The vertical stripe compensation value is set to the pixel difference; if the difference is greater than the set threshold, there is no vertical stripe phenomenon at the pixel, and the vertical stripe compensation value is set to 0. This process can be expressed as
[0027]
[0028] Wherein, cp(x,y) represents the vertical stripe compensation value of the coordinate point (x,y), and th is the set threshold;
[0029] Finally, the average value of each column compensation value is used to perform vertical stripe compensation on the pixel to remove the vertical stripes. This process can be expressed as
[0030]
[0031] Among them, I cfpn (x,y) represents the image I after vertical stripes are removed cfpn The pixel value of the coordinate point (x, y), sgn represents the sign function, and H is the image height; this step can correct the pixel value of the image column with serious vertical stripes to achieve the effect of removing vertical stripes.
[0032] b. Non-local mean filtering: Non-local mean filtering is performed on the image after vertical stripes are removed, which can be expressed as
[0033]
[0034] Among them, I nlm (x,y) represents the image I after non-local mean filtering nlm The pixel value of the coordinate point (x, y); k1 is the neighborhood radius, w(x, y, x+i, y+j) represents the weight of the pixel point (x+i, y+j) in the neighborhood to the center point (x, y), and its calculation process can be expressed as
[0035]
[0036] Where k2 is the search window radius, kernel represents the spatial kernel used to apply spatial position weights. The closer the pixel is to the center point, the greater the weight. The specific value needs to be adjusted during the optimization process. and They represent sub-graphs with (x, y) and (u, v) as the center and a radius of k2. In addition to the influence of spatial position, the closer the pixel value to the center point is, the greater the weight is. This design can effectively suppress the interference of noise on pixel values and achieve the filtering effect.
[0037] Furthermore, the step 2) specifically includes the following steps:
[0038] (1) Image linear mapping: Map the 14-bit filtered original image to the 0-255 range and convert it into an 8-bit grayscale image. The specific process is as follows:
[0039] First, the filtered original image is truncated, and its calculation process can be expressed as
[0040]
[0041] Among them, I nlm (x,y,t) and I clip (x, y, t) represent the image I at time t before and after truncation respectively nlm and image I clip The pixel value of the coordinate point (x, y), th low (t) and th high (t) are the lower and upper limits of the pixel value cutoff at time t. In order to reduce the sudden brightness change caused by scene changes, the setting of the upper and lower limits of the pixel value cutoff needs to refer to both the current frame and the previous adjacent frames. The specific calculation process can be expressed as:
[0042]
[0043] Where T represents the time window length;
[0044] Then, a linear mapping operation is performed on the filtered original image. In order to prevent excessive stretching in the scenario where the difference between the lower limit and the upper limit of the truncation is small, special processing is required for this situation. The linear mapping operation can be specifically expressed as
[0045]
[0046] Among them, I stretch is the image after linear mapping, th stretch is the set range threshold; in the linear mapping operation, if the difference between the upper and lower limits of the truncation is less than the range threshold, a fixed range threshold is used for stretching to prevent over-stretching and causing obvious noise in the picture.
[0047] (2) Adaptive histogram equalization with limited contrast:
[0048] First, the linearly mapped image is divided into several sub-images. After the histogram of each sub-image is statistically analyzed, the histogram is truncated to prevent the histogram equalization from over-enhancing the image contrast. The calculation process of the histogram truncation can be expressed as
[0049]
[0050] Among them, Hij (n) and Respectively represent the number of pixels with pixel value n in the sub-image corresponding to row index i and column index j before and after histogram truncation; th clip Represents the histogram cutoff threshold, N excess and N bin are the number of pixels and pixel values in the sub-image that exceed the histogram cutoff threshold, respectively.
[0051] Then, according to the truncated histogram, the histogram equalization mapping relationship of each subgraph is calculated, and the calculation process can be expressed as
[0052]
[0053] Among them I ij (x,y) and Respectively represent the pixel value of the coordinate point (x, y) in the sub-image with row index i and column index j before and after histogram equalization; N is the number of pixels in the sub-image.
[0054] (3) S-curve adjustment: In this step, a monotonous two-segment gamma mapping curve is designed to map pixel values and adjust the overall brightness distribution and contrast of the image. The calculation process can be expressed as:
[0055]
[0056] Among them I s (x,y) represents the image I after S-curve adjustment S The pixel value of the coordinate point (x, y), γ lpw and γ high are the two-segment stretching index parameters, and c is the coordinate parameter of the center point of the curve.
[0057] Furthermore, the main step of step 3) is to extract the high-frequency details of the image, scale them and superimpose them on the original image to enhance the image details. In the process of extracting the high-frequency details of the image, the high-frequency details are calculated for each pixel in the image. The calculation process of the high-frequency part of each pixel can be expressed as
[0058]
[0059] Among them, t hf It represents the high-frequency details of the generated image. After the high-frequency details of the image are calculated, they are scaled and superimposed on the original image to complete the detail enhancement.
[0060] Furthermore, in step 4), applying a pseudo color palette to the detail enhanced image is specifically: mapping the pixel gray value of each pixel point to the color value of the RGB three-channel to enhance the visual effect of the image; there are 10 pseudo color palettes, including "white hot", "high light", "black hot", "red hot", "iron red", "rainbow 1", "rainbow 2", "high contrast rainbow", "lava" and "Arctic", corresponding to ten different visual effects respectively.
[0061] The present invention also provides a thermal infrared image human eye comfort improvement system for executing the above method, which is composed of a plurality of modules cascaded, and guides parameter update through unsupervised training. The method is suitable for reasoning in various computing devices. The system is a cascade structure, so it can be accelerated in parallel through pipeline and other methods. The system can be divided into four modules, namely, the original image filtering module, the contrast / brightness adjustment module, the detail enhancement module and the pseudo-color palette mapping module, and the four modules can coordinate parameter adjustment and reasoning.
[0062] The original image filtering module is used to filter the original image to reduce the noise of the original image; the original image filtering module includes two submodules: time domain filtering and spatial domain filtering; the time domain filtering submodule is used to perform time domain filtering on the currently input single infrared image according to the previous frame image to remove part of the time domain noise in the image; the spatial domain filtering submodule is used to remove vertical stripes and point noise in the infrared image processed by the time domain filtering submodule;
[0063] The contrast / brightness adjustment module includes a linear mapping submodule, a contrast-limited adaptive histogram equalization submodule and an S-curve adjustment submodule; the linear mapping submodule is used to perform linear mapping on the image output by the original image filtering module in the spatial domain, thereby converting the 14-bit original image into an 8-bit grayscale image; the contrast-limited adaptive histogram equalization submodule is used to perform contrast-limited adaptive histogram equalization on the grayscale image, thereby enhancing the contrast of the image; the S-curve adjustment submodule is used to use the S-curve to further adjust the brightness and contrast of the image output by the contrast-limited adaptive histogram equalization submodule;
[0064] The detail enhancement module is used to perform neighborhood mean filtering on each pixel point of the image output by the contrast / brightness adjustment module to obtain the low-frequency part of the image, subtract the extracted low-frequency part from the original image to generate the high-frequency details of the image, and scale the high-frequency details and superimpose them on the original image for image enhancement;
[0065] The pseudo-color palette mapping module is used to apply a pseudo-color palette to the image output by the detail enhancement module to improve the viewing comfort of the image; the method for obtaining the pseudo-color palette is: uniformly selecting a number of pixel values in the range of 0-255, manually adjusting the mapped pixel values, using these points to perform cubic spline interpolation to generate a mapping curve of the pseudo-color palette, manually adjusting the mapped pixel values according to the mapping effect of the pseudo-color palette, and designing a pseudo-color palette.
[0066] Furthermore, the differential evolution algorithm is used to update all parameters to be optimized in the system to obtain better parameters. The specific steps include:
[0067] (1) Parameter population initialization: Within the upper and lower limits of each parameter to be adjusted, the parameter is randomly selected multiple times to generate the 0th generation parameter population. This process is expressed as
[0068] x ij (0) = x low (j)+rand(0,1)*(x high (j)-x low (j)),#(16)
[0069] where x ij (0) represents the value of the jth parameter in the ith individual of the parameter population of generation 0; x low (j) and x high (j) are the lower and upper limits of the parameter x(j), respectively;
[0070] (2) Parameter variation: Randomly select a vector from the current parameter population and randomly construct a difference vector to perform parameter variation. The calculation process can be expressed as
[0071] v i (g) = x r1 (g)+F*(x r2 (g)-x r3 (g)),#(17)
[0072] where v i (g) represents the individual x in the g-th generation parameter population i (g) The corresponding variant individual; x r1 (g), x r2 (g) and x r3 (g) are three different individuals randomly selected from the parameter population of the gth generation, and F is the variation factor.
[0073] (3) Parameter crossover: Each individual in the parameter population is crossed with the generated variant individual to generate a test individual. Each parameter in the test individual has a certain probability of retaining the value before the mutation. The calculation process can be expressed as:
[0074]
[0075] Among them, x ij (g) and v ij (g) are the values of the jth parameter in the i-th individual of the g-th generation parameter population before and after mutation; CR is the crossover probability factor.
[0076] (4) Parameter selection: For each individual in the parameter population, by comparing the image processing effect of the system using the individual parameters, the better parameters are selected from the original individual and the experimental individual as the next generation, and the iteration continues; the calculation process can be expressed as
[0077]
[0078] Where f(·) represents a function that measures the effect of a set of parameters. The specific calculation process can be expressed as
[0079]
[0080] in is the original image involved in the test, N is the number of test images, ISP(·) represents the image processing method of the present invention, and IQA(·) represents the image quality assessment function. The local variance method is used to evaluate the image quality of the infrared image processed by the method of the present invention, and the calculation process can be expressed as follows:
[0081]
[0082] Among them, I isp represents the infrared image processed by the method of the present invention; I isp The sub-image corresponding to row index i and column index j in the image; μ and σ represent the mean and standard deviation of the image respectively.
[0083] The beneficial effects of the present invention are:
[0084] The present invention uses a variety of mapping methods to adjust the contrast / brightness of thermal infrared images, uses a pseudo-color palette mapping method to generate a pseudo-color image of the thermal infrared image, and improves the human eye comfort of the thermal infrared image. For the original thermal infrared image collected by the infrared sensor, the original image filtering method is first used to remove part of the temporal and spatial noise to improve the quality of the original thermal infrared image. The original thermal infrared image is converted into an 8-bit grayscale image using a linear mapping method. During the conversion process, the linear mapping can retain most of the information in the original image. Through the setting of pixel value truncation and extreme difference threshold, the linear mapping method used can adaptively control the mapping range and is robust to noise interference and low extreme difference scenes. On the grayscale image generated by the conversion, the mapping function generated by the adaptive histogram equalization with limited contrast is used to improve the contrast of the image, and the S-curve mapping function is used to further fine-tune the contrast / brightness of the image. After extracting the high-frequency details of the image for detail enhancement, the pseudo-color palette mapping method is used to add different pseudo-color effects to the thermal infrared image. The differential evolution algorithm is used to tune the parameters in the proposed method, thereby obtaining a relatively excellent thermal infrared image enhancement effect. BRIEF DESCRIPTION OF THE DRAWINGS
[0085] Figure 1 Flow chart of the method of the present invention.
[0086] Figure 2 It is the original image filtering module.
[0087] Figure 3 It is the contrast / brightness adjustment module.
[0088] Figure 4 Flowchart of parameter tuning using differential evolution algorithm.
[0089] Figure 5 For comparison with existing advanced infrared image human eye comfort improvement algorithms.
[0090] Figure 6 This is the image effect after pseudo color palette mapping. DETAILED DESCRIPTION
[0091] The specific implementation modes of the present invention are further described below in conjunction with the accompanying drawings.
[0092] like Figure 1 As shown, the present invention provides a thermal infrared image processing method for improving human eye comfort, and the method relies on a thermal infrared image human eye comfort improvement system. The specific steps of the method of the present invention are as follows:
[0093] (1) Using the original image filtering module to filter the original thermal infrared image, using the time domain filtering and spatial domain filtering submodules to perform time domain and spatial domain filtering on the 14-bit grayscale image, respectively, to suppress the noise contained in the original image and improve the image quality; the time domain filtering submodule is used to perform time domain filtering on the currently input single infrared image according to the previous frame image to remove part of the time domain noise in the image; the spatial domain filtering submodule is used to remove vertical stripes and point noise in the infrared image processed by the time domain filtering submodule;
[0094] (2) using the contrast / brightness adjustment module to process the image output by the original image filtering module, and adaptively adjusting the contrast and brightness of the entire image according to the temporal and spatial characteristics of the image, so as to improve the overall perception of the image; using the linear mapping submodule to perform linear mapping on the image output by the original image filtering module in the spatial domain, and converting the 14-bit image into an 8-bit grayscale image; the contrast-limited adaptive histogram equalization submodule is used to perform contrast-limited adaptive histogram equalization on the grayscale image, so as to enhance the contrast of the image; the S-curve adjustment submodule is used to further adjust the brightness and contrast of the image output by the contrast-limited adaptive histogram equalization submodule using the S-curve;
[0095] (3) The image output by the contrast / brightness adjustment module is processed by a detail enhancement module to calculate the high-frequency details of the image, and the high-frequency details are scaled and superimposed on the original image to enhance the image details. Preferably, the high-frequency image scaling factor is set to 0.1. The high-frequency details of the image are calculated by performing neighborhood mean filtering on each pixel point of the image to obtain the low-frequency part of the image, and subtracting the extracted low-frequency part from the original image to generate the high-frequency details of the image.
[0096] (4) Apply a pseudo-color palette to the image output by the detail enhancement module using a pseudo-color palette mapping module, and select a suitable pseudo-color palette to color the grayscale image according to the user's preference and specific usage scenario, thereby obtaining a pseudo-color image. The pseudo-color palette is obtained by uniformly selecting a number of pixel values in the range of 0-255, manually adjusting the pixel values after mapping, using these points to perform cubic spline interpolation to generate a mapping curve of the pseudo-color palette, and manually adjusting the mapped pixel values according to the mapping effect of the pseudo-color palette, thereby designing a pseudo-color palette.
[0097] The original image is filtered using the original image filtering module, such as Figure 2 As shown, the specific reasoning process includes the following steps:
[0098] (1) The image is processed using the time domain filtering submodule. The specific method is as follows: a weight matrix is generated based on the current frame and the previous frame, and the weighted sum of the two frames is calculated to suppress the time domain noise. The specific implementation is based on the following formula:
[0099] I tfilt (t)=(1-W(t))☉I raw (t)+W(t)⊙I raw (t-1),#(1)
[0100] Among them, W(t) represents the weight matrix at time t, I tfilt (t) represents the image after time domain filtering at time t, I raw (t) and I raw (t-1) are the original image of the current frame and the original image of the previous frame respectively, ⊙ is the Hadamard product of the matrix;
[0101] The calculation process of the weight matrix W(t) is expressed as
[0102] W(t)=α*exp(-|I raw (t)-I raw (t-1) / h),#(2)
[0103] Wherein, α and h are parameters. By adjusting the values of the parameters α and h, the weights of the pixel values of the current frame and the previous frame in the weighted summation are controlled. Preferably, α=70 and h=150 are set in formula (2);
[0104] (2) Spatial filtering submodule, used to perform vertical stripe removal and non-local mean filtering steps:
[0105] The vertical stripe removal is specifically as follows:
[0106] First, the input image is subjected to a Gaussian filter with a horizontal window, expressed as:
[0107]
[0108] Among them, kernel represents the Gaussian kernel used in Gaussian filtering, R is the radius of the Gaussian kernel, and I tfilt (x+r,y) represents image I tfilt The pixel value of the coordinate point (x+r,y), I gfilt (x,y) represents the image I after horizontal window Gaussian filtering gfilt The pixel value of the coordinate point (x, y); preferably, R=4 in formula (3).
[0109] Then, the image after Gaussian filtering is compared with the original image to suppress the detected vertical stripes. Specifically, the pixel difference of each pixel in the image before and after Gaussian filtering is calculated. If the difference is less than the set threshold, it is determined that the pixel value of the pixel is similar to the horizontal neighborhood, and there is a vertical stripe phenomenon. The vertical stripe compensation value is set to the pixel difference; if the difference is greater than the set threshold, there is no vertical stripe phenomenon at the pixel, and the vertical stripe compensation value is set to 0. This process is expressed as
[0110]
[0111] Wherein, cp(x, y) represents the vertical stripe compensation value of the coordinate point (x, y), and th is the set threshold value; preferably, th=30 in formula (4).
[0112] Finally, the average value of each column compensation value is used to perform vertical stripe compensation on the pixel to remove the vertical stripes, which is expressed as
[0113]
[0114] Among them, I cfpn (x,y) represents the image I after vertical stripes are removed cfpn The pixel value of the coordinate point (x, y), sgn represents the sign function, and H is the image height.
[0115] The non-local mean filtering is specifically to perform non-local mean filtering on the image after the vertical stripes are removed, which is expressed as
[0116]
[0117] Among them, I nlm (x,y) represents the image I after non-local mean filtering nlm The pixel value of the coordinate point (x, y); k1 is the neighborhood radius, w(x, y, x+i, y+j) represents the weight of the pixel point (x+i, y+j) in the neighborhood to the center point (x, y), and its calculation process is expressed as
[0118]
[0119] Where k2 is the search window radius, kernel represents the spatial kernel used to apply spatial position weights. The closer the pixel is to the center point, the greater the weight. The specific value needs to be adjusted during the optimization process. and They represent sub-images centered at (x, y) and (u, v) with a radius of k2. Preferably, k1 in formula (6) is set to 9, and k2 in formula (7) is set to 3.
[0120] The image output by the original image filtering module is processed using the contrast / brightness adjustment module, such as Figure 3 As shown, the specific reasoning process steps include the following:
[0121] (1) Use the linear mapping submodule to process the image, map the input 14-bit image to the 0-255 range, and convert it into an 8-bit grayscale image; the specific process is as follows:
[0122] First, the filtered original image is truncated, and the calculation process is expressed as
[0123]
[0124] Among them, I nlm (x,y,t) and I clip (x, y, t) represent the image I at time t before and after truncation respectively nlm and image I clip The pixel value of the coordinate point (x, y), th low (t) and th high (t) are the lower and upper limits of the pixel value at time t respectively; the calculation formula of the upper and lower limits of the pixel value is expressed as:
[0125]
[0126] Where T represents the time window length;
[0127] Then, a linear mapping operation is performed on the filtered original image, which is specifically expressed as
[0128]
[0129] Among them, I stretch is the image after linear mapping, th stretch is the set extreme value threshold;
[0130] As a preferred method, set th in formula (11) stretch =1000;
[0131] (2) The image is processed using the contrast-limited adaptive histogram equalization submodule to improve the contrast of the input image. The specific method is as follows:
[0132] Firstly, the linearly mapped image is evenly divided into several sub-images. After the histogram of each sub-image is statistically analyzed, the histogram is truncated to prevent the histogram equalization from over-enhancing the image contrast.
[0133] The calculation process of histogram truncation is expressed as
[0134]
[0135] Among them, H ij (n) and Respectively represent the number of pixels with pixel value n in the sub-image corresponding to row index i and column index j before and after histogram truncation; th clip Represents the histogram cutoff threshold, N excess and N bin are the number of pixels and pixel values in the sub-image that exceed the histogram cutoff threshold, respectively;
[0136] Then, according to the truncated histogram, the histogram equalization mapping relationship of each sub-image is calculated, and the calculation process is expressed as
[0137]
[0138] Among them I ij (x,y) and Respectively represent the pixel value of the coordinate point (x, y) in the sub-image with row index i and column index j before and after histogram equalization; N is the number of pixels in the sub-image; as a preferred method, set th in formula (12) clip =600;
[0139] (3) The S-curve adjustment submodule is used to process the image and the pixel values are mapped using a monotonous two-segment gamma mapping curve to complete the adjustment of the overall brightness distribution and contrast of the image. The calculation process is expressed as
[0140]
[0141] Among them I s (x,y) represents the image I after S-curve adjustment S The pixel value of the coordinate point (x, y), γ low and γ high is the two-stage stretching index parameter, and c is the coordinate parameter of the center point of the curve. This module can fine-tune the overall brightness and contrast of the image. As a preferred method, set γ in formula (14) low =0.6,γ high =1.1, c=0.25.
[0142] In the process of processing the image output by the contrast / brightness adjustment module using the detail enhancement module, the high-frequency details of the image are generated by subtracting the extracted low-frequency part from the original image, which is expressed as
[0143]
[0144] Among them, I hf Represents the high-frequency details of the generated image.
[0145] A pseudo-color palette mapping module is used to apply a pseudo-color palette to the image output by the detail enhancement module. The specific method is: the pixel grayscale value of each pixel is mapped to the color value of the RGB three-channel to enhance the visual effect of the image; there are 10 pseudo-color palettes, including "white hot", "high light", "black hot", "red hot", "iron red", "rainbow 1", "rainbow 2", "high contrast rainbow", "lava" and "arctic", corresponding to ten different visual effects respectively.
[0146] The differential evolution algorithm is used to update all parameters to be optimized in the system, so as to obtain better parameters. The differential evolution parameter tuning algorithm process used in the present invention is as follows: Figure 4 As shown,
[0147] After initializing and generating the 0th generation parameter population, the optimization loop is entered. Each loop executes parameter population mutation and crossover. The better parameter population is selected and retained according to the quality of the generated image, and the optimal solution is finally obtained. The update steps of the parameters to be optimized are as follows:
[0148] (1) Parameter population initialization: Within the upper and lower limits of each parameter to be adjusted, the parameter is randomly selected multiple times to generate the 0th generation parameter population. This process is expressed as
[0149] x ij (0) = x low (j)+rand(0,1)*(x high (jx low ( / )),#(16)
[0150] where x ij (0) represents the value of the jth parameter in the ith individual of the parameter population of generation 0; x low (j) and x high (j) are the lower and upper limits of the parameter x(j), respectively;
[0151] (2) Parameter variation: Randomly select a vector from the current parameter population and randomly construct a difference vector to perform parameter variation. The calculation process can be expressed as
[0152] v i (g) = x r1 (g)+F*(x r2 (g)-x r3 (g)),#(17)
[0153] where v i (g) represents the individual x in the g-th generation parameter population i (g) The corresponding variant individual; x r1 (g), x r2 (g) and x r3(g) are three different individuals randomly selected from the parameter population of the gth generation, and F is the variation factor.
[0154] (3) Parameter crossover: Each individual in the parameter population is crossed with the generated variant individual to generate a test individual. Each parameter in the test individual has a certain probability of retaining the value before the mutation. The calculation process can be expressed as:
[0155]
[0156] Among them, x ij (g) and v ij (g) are the values of the jth parameter in the i-th individual of the g-th generation parameter population before and after mutation; CR is the crossover probability factor.
[0157] (4) Parameter selection: For each individual in the parameter population, by comparing the image processing effect of the system using the individual parameters, the better parameters are selected from the original individual and the experimental individual as the next generation, and the iteration continues; the calculation process can be expressed as
[0158]
[0159] Where f(·) represents a function that measures the effect of a set of parameters. The specific calculation process can be expressed as
[0160]
[0161] in is the original image involved in the test, N is the number of test images, ISP(·) represents the image processing method of the present invention, and IQA(·) represents the image quality assessment function. The local variance method is used to evaluate the image quality of the infrared image processed by the method of the present invention, and the calculation process can be expressed as follows:
[0162]
[0163] Among them, I isp represents the infrared image processed by the method of the present invention; I isp The sub-image corresponding to row index i and column index j in the image; μ and σ represent the mean and standard deviation of the image respectively.
[0164] Preferably, F=0.6 in formula (17) and CR=0.6 in formula (18) are set.
[0165] like Figure 5 This is a comparison result between the method of the present invention and the existing advanced infrared image human eye comfort improvement algorithm. It can be seen from the figure that the method of the present invention has a better visual improvement effect.
[0166] like Figure 6 The figure shows the effect of applying a pseudo-color palette to an image. It can be seen from the figure that the color palette proposed by the present invention has a better visual effect.
[0167] The above are only specific implementation methods of the present invention, which cannot be used to limit the scope of the present invention. Equivalent changes made by ordinary technicians in this technical field based on this creation, as well as changes known to technicians in this field, should still fall within the scope of the present invention.
Claims
1. A thermal infrared image processing method for improving human eye comfort, characterized in that: Here are the steps: 1) Original image filtering: filtering the original image to reduce the noise of the original image; the filtering includes time domain filtering and spatial domain filtering; the specific method of the time domain filtering is: for a single infrared image input, the current frame image is subjected to time domain filtering according to the previous frame image to remove part of the time domain noise in the image; the spatial domain filtering is used to remove the vertical stripes and point noise in the infrared image after the time domain filtering; 2) Contrast / brightness adjustment: First, the filtered image of the original image is linearly mapped in the spatial domain, thereby converting the 14-bit original image into an 8-bit grayscale image; then, the grayscale image is subjected to contrast-limited adaptive histogram equalization, thereby enhancing the contrast of the image; finally, the brightness and contrast of the image are further adjusted using an S curve; 3) Detail enhancement: First, neighborhood mean filtering is performed on each pixel of the contrast / brightness adjusted image to obtain the low-frequency part of the image, and the extracted low-frequency part is subtracted from the original image to generate the high-frequency details of the image. The high-frequency details are scaled and superimposed on the original image for image enhancement; 4) Pseudo-color palette mapping: applying a pseudo-color palette to the image after detail enhancement to improve the viewing comfort of the image; the method for obtaining the pseudo-color palette is: uniformly selecting a number of pixel values in the range of 0-255, manually adjusting the mapped pixel values, using these points to perform cubic spline interpolation to generate a mapping curve of the pseudo-color palette, manually adjusting the mapped pixel values according to the mapping effect of the pseudo-color palette, and designing a pseudo-color palette.
2. The thermal infrared image processing method for improving human eye comfort as claimed in claim 1, characterized in that: The step 1) specifically comprises the following steps: (1) Time domain filtering: The current infrared image is filtered in the time domain according to the previous image. The process is expressed as: tfilt (t)=1-W(t))⊙I raw (t)+W()⊙I raw (-1),#(1) Among them, W(t) represents the weight matrix at time t, I tfilt (t) represents the image after time domain filtering at time t, I raw () and I raw (-1) are the original image of the current frame and the original image of the previous frame respectively, ⊙ is the Hadamard product of the matrix; The calculation process of the weight matrix W(t) is expressed as W(t) = *exp(-|I raw (t)- raw (t-1)| / h),#(2) Among them, α and h are parameters. By adjusting the values of parameters α and h, the weights of the pixel values of the current frame and the previous frame in the weighted summation are controlled; (2) Spatial filtering: including vertical stripe removal and non-local mean filtering; a. Remove vertical stripes; First, the image is subjected to a Gaussian filter with a horizontal window. The calculation process is expressed as Among them, kernel represents the Gaussian kernel used in Gaussian filtering, R is the radius of the Gaussian kernel, and I tfilt (+r,y) represents image I tfilt The pixel value of the midpoint (+r, y), I gfilt (x,y) represents the image I after horizontal window Gaussian filtering gfilt The pixel value of the midpoint (,y); Next, the pixel difference of each pixel in the image before and after Gaussian filtering is calculated. If the difference is less than the set threshold, it is determined that the pixel value of the pixel is similar to the horizontal neighborhood, and there is a vertical stripe phenomenon. The vertical stripe compensation value is set to the pixel difference; if the difference is greater than the set threshold, there is no vertical stripe phenomenon at the pixel, and the vertical stripe compensation value is set to 0. The process is expressed as Where cp(x,y) represents the vertical stripe compensation value of the coordinate point (,y), and th is the set threshold; Finally, the average value of each column compensation value is used to perform vertical stripe compensation on the pixel to remove the vertical stripes, which is expressed as Among them, I cfpn (x,y) represents the image I after vertical stripes are removed cfpn The pixel value of the coordinate point (,y), sgn represents the sign function, and H is the image height; b. Non-local mean filtering: Non-local mean filtering is performed on the image after vertical stripes are removed, which is expressed as Among them, I nlm (x,y) represents the image I after non-local mean filtering nlm k1 is the neighborhood radius, w(x,y,x+i,y+j) represents the weight of the pixel point (x+i,y+j) in the neighborhood to the center point (,y), and its calculation process is expressed as Where k2 is the search window radius, kernel represents the spatial kernel used to apply spatial position weights, and the closer the pixel is to the center point, the greater the weight; and They represent sub-images centered at (,y) and (,v) with a radius of k2.
3. The thermal infrared image processing method for improving human eye comfort as claimed in claim 1, characterized in that: The step 2) specifically comprises the following steps: (1) Image linear mapping: Map the 14-bit filtered original image to the 0-255 range and convert it into an 8-bit grayscale image. The specific process is as follows: First, the filtered original image is truncated, and the calculation process is expressed as Among them, I nlm (x,y,t) and I clip (,y,t) respectively represent the image I before and after truncation at time t nlm and image I clip The pixel value of the coordinate point (,y), th low () and th high () are the lower and upper limits of pixel value truncation at time t respectively; The calculation formula for the upper and lower limits of pixel value truncation is expressed as Where T represents the time window length; Then, a linear mapping operation is performed on the filtered original image, which is specifically expressed as Among them, I stretch is the image after linear mapping, th stretch is the set extreme value threshold; (2) Adaptive histogram equalization with limited contrast: First, the linearly mapped image is evenly divided into several sub-images, and the histogram of each sub-image is statistically analyzed and then the histogram is truncated; The calculation process of histogram truncation is expressed as Among them, H ij ()and Respectively represent the number of pixels with pixel value n in the sub-image corresponding to row index i and column index j before and after histogram truncation; th clip Represents the histogram cutoff threshold, N excess and N bin are the number of pixels and pixel values in the sub-image that exceed the histogram cutoff threshold, respectively; Then, according to the truncated histogram, the histogram equalization mapping relationship of each sub-image is calculated, and the calculation process is expressed as Among them I ij (,y) and Respectively represent the pixel value of the coordinate point (,y) in the sub-image with row index i and column index j before and after histogram equalization; N is the number of pixels in the sub-image; (3) S-curve adjustment: The pixel values are mapped using a monotonous two-segment gamma mapping curve to adjust the overall brightness distribution and contrast of the image. The calculation process is expressed as Among them I S (x,y) represents the image I after S-curve adjustment S The pixel value of the midpoint (,y), γ low and γ high are the two-segment stretching index parameters, and c is the coordinate parameter of the center point of the curve.
4. The thermal infrared image processing method for improving human eye comfort according to claim 1, characterized in that: In step 3), the high-frequency details of the image are generated by subtracting the extracted low-frequency part from the original image, which is expressed as Among them, I hf Represents the high-frequency details of the generated image.
5. The thermal infrared image processing method for improving human eye comfort as claimed in claim 1, characterized in that: In step 4), applying a pseudo color palette to the detail enhanced image specifically includes mapping the pixel gray value of each pixel point to the color value of the RGB three channels to enhance the visual effect of the image; there are 10 pseudo color palettes, including "white hot", "high light", "black hot", "red hot", "iron red", "rainbow 1", "rainbow 2", "high contrast rainbow", "lava" and "arctic", corresponding to ten different visual effects respectively.
6. A thermal infrared image human eye comfort improvement system for executing any of the methods described in steps 1-5, characterized in that: It includes original image filtering module, contrast / brightness adjustment module, detail enhancement module and pseudo color palette mapping module; The original image filtering module is used to filter the original image to reduce the noise of the original image; the original image filtering module includes two submodules: time domain filtering and spatial domain filtering; the time domain filtering submodule is used to perform time domain filtering on the currently input single infrared image according to the previous frame image to remove part of the time domain noise in the image; The spatial domain filtering submodule is used to remove vertical stripes and point noise in the infrared image processed by the temporal domain filtering submodule; The contrast / brightness adjustment module includes a linear mapping submodule, an adaptive histogram equalization submodule for limiting contrast, and an S-curve adjustment submodule; The linear mapping submodule is used to perform linear mapping on the image output by the original image filtering module in the spatial domain, thereby converting the 14-bit original image into an 8-bit grayscale image; the contrast-limited adaptive histogram equalization submodule is used to perform contrast-limited adaptive histogram equalization on the grayscale image, thereby enhancing the contrast of the image; the S-curve adjustment submodule is used to further adjust the brightness and contrast of the image output by the contrast-limited adaptive histogram equalization submodule using the S-curve; The detail enhancement module is used to perform neighborhood mean filtering on each pixel point of the image output by the contrast / brightness adjustment module to obtain the low-frequency part of the image, subtract the extracted low-frequency part from the original image to generate the high-frequency details of the image, and scale the high-frequency details and superimpose them on the original image for image enhancement; The pseudo-color palette mapping module is used to apply a pseudo-color palette to the image output by the detail enhancement module to improve the viewing comfort of the image; the method for obtaining the pseudo-color palette is: uniformly selecting a number of pixel values in the range of 0-255, manually adjusting the mapped pixel values, using these points to perform cubic spline interpolation to generate a mapping curve of the pseudo-color palette, manually adjusting the mapped pixel values according to the mapping effect of the pseudo-color palette, and designing a pseudo-color palette.
7. The system according to claim 6, characterized in that Use the differential evolution algorithm to update all parameters to be optimized in the system. The specific steps are: (1) Parameter population initialization: Within the upper and lower limits of each parameter to be adjusted, the parameter is randomly selected multiple times to generate the 0th generation parameter population. This process is represented by x ij (0)= low (j)+rand(0,1)*(x high (j)- low (j)),#(16) where x ij (0) represents the value of the jth parameter in the ith individual of the parameter population of generation 0; x low (j) and x high (j) are the lower and upper limits of the parameter x(j), respectively; (2) Parameter variation: Randomly select a vector from the current parameter population and randomly construct a difference vector to perform parameter variation. The calculation process is represented by v i (g)= r1 (g)+*(x r2 (g)- r3 (g)),#(17) where v i (g) represents the individual x in the g-th generation parameter population i (g) The corresponding variant individual; x r1 (g), x r2 (g) and x r3 (g) are three different individuals randomly selected from the parameter population of the gth generation, and F is the variation factor; (3) Parameter crossover: Cross each individual in the parameter population with the variant individual it generates to generate the test individual u ij (g), the calculation process is expressed as Among them, x ij (g) and v ij (g) is the value of the jth parameter in the i-th individual of the g-th generation parameter population before and after the mutation; CR is the crossover probability factor; (4) Parameter selection: For each individual in the parameter population, by comparing the image processing effect of the system using the individual parameters, the better parameters are selected from the original individual and the experimental individual as the next generation, and the iteration continues; the calculation process is expressed as Where f(·) represents a function that measures the effect of a set of parameters. The specific calculation process can be expressed as in is the original image involved in the test, N is the number of test images, ISP(·) represents the image processing method of the present invention, and IQA(·) represents the image quality assessment function. The local variance method is used to evaluate the image quality of the infrared image processed by the method of the present invention, and the calculation process is expressed as follows: Among them I isp represents the infrared image processed by the method of the present invention; I isp The sub-image corresponding to row index i and column index j in the image; μ and σ represent the mean and standard deviation of the image respectively.
Citation Information
Cited By
Infrared image brightness enhancement method
CN120563390A
