Color image fusion algorithm for enhancing wave band difference by using human eye color visual difference

Abstract

The invention relates to a color image fusion algorithm for enhancing band difference by using human eye color vision difference, and belongs to the technical field of image processing. The fusion algorithm comprises the following steps: extracting and fusing dual-band spatial information; wave band information difference extraction; wave band difference information is mapped to a color space; carrying out adaptive double-histogram equalization on the mapping saturation; and generating and converting an LAB space color fusion image. In order to solve the problem of dual-band image fusion and how to characterize or enhance the difference information of bands is always a hot spot and a difficult point of research in the field, the method provided by the invention utilizes the visual difference of human eyes to colors to enhance the band difference of a fused image in an LAB color space, and carries out balanced regulation and control; the color contrast of the image can be enhanced by using color visual difference caused by wave band difference while high-frequency space information of the fused image is reserved, so that the color fused image which is more suitable for observation and detection and has good scene adaptability is generated.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a color image fusion algorithm that utilizes the differences in human eye color vision to enhance band differences. Background Technology

[0002] Dual-band infrared imaging technology, by simultaneously acquiring infrared radiation information from two atmospheric windows, combines the advantages of two single-band imaging techniques. Through this complementary approach, dual-band infrared imaging technology can improve the adaptability of infrared imaging equipment to various complex environmental conditions and increase the success rate of target detection and identification. How to integrate and display different band and spatial information using image fusion algorithms to present a better visual effect is of great significance for fully leveraging the advantages of dual-band infrared imaging technology.

[0003] Color fusion primarily utilizes the human eye's color vision capabilities to convey image information. The process of color fusion between two images is a dimensionality-upgrading process, with a focus on utilizing high-dimensional space. Infrared dual-band image fusion falls into this category, requiring the use of as many dimensions of visual space as possible to enhance the display of information from both bands. Current classic color fusion algorithms, such as the TNO algorithm, NRL algorithm, color transfer algorithms, and algorithms based on hue and saturation color spaces, also perform color fusion through color space. However, the question of how to correctly use color space to map band information has not been thoroughly investigated. The result is that the fusion effect is unstable, and the band information represented by the color information is unclear. Significant band differences may not produce a correspondingly large visual effect in the fused image; that is, these algorithms may lead to band information confusion after mapping, resulting in information distortion in the color fused image. Due to the lack of a scientifically effective mapping theory, these algorithms typically have low scene adaptability and cannot always produce color fusion effects that are beneficial for observation. Therefore, it is necessary to conduct in-depth research on the mapping relationship between band differences and color space. How to better utilize color space so that the band difference information is not distorted during the color image fusion process and generate a color fusion image that is more conducive to human visual observation and detection is an urgent problem to be solved in this field. Summary of the Invention

[0004] The problem this invention aims to solve is that traditional algorithms often lack a clear mapping relationship between band differences and color space, leading to distortion of band difference information during color fusion. This invention provides a color image fusion algorithm that leverages human visual color differences to enhance band differences. This invention ensures that the color visual differences in the fused image are positively correlated with their corresponding band differences, thus achieving image color enhancement. To increase the algorithm's scene adaptability, a dual histogram equalization algorithm is introduced to balance and control the saturation of the color-fused image. Ultimately, this achieves the goal of enhancing band differences using human visual color differences.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A color image fusion algorithm that leverages differences in human color vision to enhance band differences includes the following steps:

[0007] Step (1), dual-band spatial information extraction and fusion: extract the spatial information of the two band infrared images and perform grayscale fusion;

[0008] Step (2), band information difference extraction: compare the gray values ​​of the two bands, and use the difference in gray values ​​as the band difference information to extract the difference information of band I and band II respectively.

[0009] Step (3), mapping band difference information to color space: mapping band difference information to the hue and saturation components of color space through color curves in color space, so that the band difference information is proportional to the visual difference of human eyes;

[0010] Step (4), adaptive dual histogram equalization of saturation mapping: calculate the mean value of saturation after mapping, and use the mean value as the dividing line to calculate the histogram of the part of pixel gray value less than the mean value and the histogram of the part of pixel gray value greater than the mean value respectively. The part less than the mean value is kept unchanged, and then the part greater than the mean value is subjected to histogram equalization.

[0011] Step (5), generation and conversion of LAB (Lightness, a, b) space color fusion image: the black and white fusion image generated in step (1) is used as the L component of the LAB space color fusion image; then, based on the hue information in step (3) and the saturation information after equalization in step (4), the red-green component A and the blue-yellow component B of the LAB space color fusion image are calculated; finally, the LAB space color image is converted to RGB space to obtain the RGB space color fusion image.

[0012] Furthermore, preferably, in step (1), grayscale fusion can be performed using mean fusion or other suitable fusion algorithms.

[0013] Furthermore, preferably, in step (2), the difference in grayscale values ​​is used as band difference information to extract band I difference information and band II difference information respectively. The specific method is as follows:

[0014] t(x,y)=LW(x,y)-MW(x,y),if LW(x,y)≥MW(x,y)

[0015] t(x,y)=MW(x,y)-LW(x,y),if MW(x,y)>LW(x,y)

[0016] In the formula, MW(x,y) and LW(x,y) are grayscale images of band I and band II, respectively, t(x,y) is the band difference information, and x and y represent the pixel position coordinates.

[0017] Furthermore, preferably, the specific method of step (3) is as follows:

[0018] First, perform tone mapping:

[0019] θ(x,y)=θ0, if MW(x,y)>LW(x,y)

[0020] θ(x,y)=θ0+π,if LW(x,y)≥MW(x,y)

[0021] In the formula, θ0 is the initial hue, which can be selected from 0 to 2π; θ(x,y) is the hue mapped to the color space;

[0022] Then, saturation mapping is performed, using the following formula:

[0023] r(x,y)=ct(x,y),if LW(x,y)≥MW(x,y)

[0024] r(x,y)=cy(x,y),if MW(x,y)>LW(x,y)

[0025] In the formula, r(x,y) is the saturation mapped to the color space, and c is the saturation mapping coefficient.

[0026] Furthermore, preferably, the specific method of step (4) is as follows:

[0027] First, calculate the mean and histogram of saturation:

[0028]

[0029] H(i) = n i

[0030] In the formula, mean_r(x,y) represents the mean saturation, M and N represent the total number of pixels in the x and y directions, respectively, i is the grayscale value, ranging from 0 to 255, H(i) is the histogram of saturation, and n i This represents the total number of pixels with a grayscale value of i. The histogram is divided into two parts: those below the mean and those above the mean, using the average value as the dividing line.

[0031] H_low(i l )=H(i≤mean_r)

[0032] H_high(i h )=H(i>mean_r)

[0033] In the formula, H_low(i l H_high(i) is a histogram of the portion of grayscale below the mean saturation. h (i) is a histogram of the portion of grayscale values ​​higher than the mean. l Gray values ​​below the mean, ranging from 0 to mean_r, i h Gray values ​​above the mean range from (mean_r+1) to 255. Values ​​below the mean are not processed. Then, the cumulative histogram of the values ​​above the mean is calculated, and histogram equalization is performed.

[0034]

[0035] if r(x,y)=i h r_out(x,y)=HE(i h )

[0036] In the formula, CDF_high(i h HE(i) is the cumulative histogram of the portion above the mean. h (i) represents the histogram-equilibrated i h The corresponding new grayscale value, 'a' is the saturation control factor used to limit the maximum saturation value after equalization, and 'r_out(x,y)' is the final saturation image.

[0037] Furthermore, preferably, the specific method of step (5) is as follows:

[0038] The grayscale fusion image generated in step (1) is used as the L component of the LAB space color fusion image; then, the hue θ(x,y) generated in step (3) and the equalized saturation r_out(x,y) generated in step (4) are mapped to the A and B components of the LAB space color fusion image, as follows:

[0039] L(x,y)=K1×GF(x,y)

[0040] A(x,y)=K2×r_out(x,y)cos(θ(x,y))

[0041] B(x,y)=K3×r_out(x,y)sin(θ(x,y))

[0042] In the formula, L(x,y) is the luminance component L of the LAB space color fusion image, GF(x,y) is the grayscale fusion image obtained in step (1), A(x,y) is the red-green component A of the LAB space color fusion image, B(x,y) is the blue-yellow component B of the LAB space color fusion image, and K1, K2, and K3 are color space mapping coefficients. Finally, the LAB space color image is converted to RGB space to obtain the R, G, and B components of the color fusion image, which are used for image transmission and display.

[0043] In this invention, grayscale fusion can be achieved using mean fusion or other suitable fusion algorithms. Grayscale fusion is mainly aimed at preserving the high-frequency spatial information of the two bands. It can be performed using methods based on multi-scale decomposition, sparse representation, and spatial domain fusion.

[0044] In this invention, 'a' is a saturation control factor used to limit the maximum saturation value after equalization. Based on experience, it can be set to 96 to limit the final output fused image from oversaturation, which could lead to the loss of spatial details.

[0045] In this invention, c is the saturation mapping coefficient, which is used to adjust the linear slope of the band difference mapping to saturation, and can be set to 1.

[0046] In this invention, θ0 is the initial hue, which can be selected from 0 to 2π to adjust the hue of the fused image. Based on experience, it can be set to 0.3π.

[0047] In this invention, K1, K2 and K3 are color space mapping coefficients, which can be set according to the selected color space scale range. For the standard CIELAB color space, L takes the value of 0 to 100, and A and B take the values ​​of -127 to +127. K1 can be set to 0.39, and K2 and K3 can be set to 0.5.

[0048] A computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to perform the steps of the color image fusion algorithm for enhancing band differences by utilizing differences in human eye color vision as described in this invention.

[0049] Compared with the prior art, the beneficial effects of this invention are as follows:

[0050] Most current dual-band color image fusion techniques operate within the RGB or HSV color spaces, while the technique proposed in this invention operates within the LAB color space. Compared to other commonly used color spaces, the LAB space offers better color accuracy, and a greater difference in LAB space distance can produce a stronger visual difference effect.

[0051] Meanwhile, traditional color image fusion techniques directly or indirectly transform and map dual-band information to the three channels of the color space through different methods. Although colorization is achieved, the relationship between color information and band information is unclear. Since the two bands often contain many similar or identical scene features, previous color fusion methods are prone to color casts, making it difficult to highlight the band differences of interest to the observer. The key to dual-band fusion technology lies in how to represent or enhance band difference information through fusion. This technology has thoroughly studied the mapping relationship between band differences and the color space. The proposed mapping method ensures that the human eye's color visual difference in the fused image is proportional to the corresponding band difference, guaranteeing the order preservation of the band difference mapping. For parts without difference, a colorless grayscale image is represented, ultimately achieving enhancement of band differences through human eye's color visual difference.

[0052] Furthermore, existing dual-band color image fusion techniques do not fully consider the significant variations in band differences across different application scenarios, only achieving good color image visual effects when the band differences within a scene are sufficiently large. This invention introduces a color adaptive adjustment mechanism into traditional color image fusion methods. It uses dual histograms to divide band differences into background and target, and then performs histogram equalization on the target's histogram, further enhancing the visual difference effect caused by the target's band differences in color image fusion. Ultimately, this invention, while preserving the mapping order of band difference information, fully utilizes the differences in human color vision to enhance band differences, achieving a color fusion effect with better visual perception and wider scene adaptability. Attached Figure Description

[0053] Figure 1 This is a flowchart of the color image fusion algorithm of the present invention that utilizes the color vision differences of the human eye to enhance the differences in bands;

[0054] Figure 2 These are infrared mid-wave (left) and mid-to-long-wave (right) images acquired using a dual-band thermal imager in four different scenarios.

[0055] Figure 3 This is a schematic diagram illustrating the effect of blending color images from four different scenes. Detailed Implementation

[0056] The present invention will now be described in further detail with reference to the embodiments.

[0057] Those skilled in the art will understand that the following embodiments are for illustrative purposes only and should not be construed as limiting the scope of the invention. Where specific techniques or conditions are not specified in the embodiments, they should be performed in accordance with the techniques or conditions described in the literature in the field or according to the product manual.

[0058] Example 1

[0059] like Figure 1 As shown, a color image fusion algorithm that enhances band differences by utilizing the differences in human color vision includes the following steps:

[0060] Step (1), dual-band spatial information extraction and fusion: extract the spatial information of the two band infrared images and perform grayscale fusion;

[0061] Step (2), band information difference extraction: compare the gray values ​​of the two bands, and use the difference in gray values ​​as the band difference information to extract the difference information of band I and band II respectively.

[0062] Step (3), mapping band difference information to color space: mapping band difference information to the hue and saturation components of color space through color curves in color space, so that the band difference information is proportional to the visual difference of human eyes;

[0063] Step (4), adaptive dual histogram equalization of saturation mapping: calculate the mean value of saturation after mapping, and use the mean value as the dividing line to calculate the histogram of the part of pixel gray value less than the mean value and the histogram of the part of pixel gray value greater than the mean value respectively. The part less than the mean value is kept unchanged, and then the part greater than the mean value is subjected to histogram equalization.

[0064] Step (5), generation and conversion of LAB space color fusion image: the black and white fusion image generated in step (1) is used as the L component of LAB space color fusion image; then, based on the hue information in step (3) and the saturation information after equalization in step (4), the red-green component A and blue-yellow component B of LAB space color fusion image are calculated; finally, the color image in LAB space is converted to RGB space to obtain RGB space color fusion image.

[0065] Example 2

[0066] like Figure 1 As shown, a color image fusion algorithm that enhances band differences by utilizing the differences in human color vision includes the following steps:

[0067] Step (1), dual-band spatial information extraction and fusion: extract the spatial information of the two band infrared images and perform grayscale fusion;

[0068] Step (2), band information difference extraction: compare the gray values ​​of the two bands, and use the difference in gray values ​​as the band difference information to extract the difference information of band I and band II respectively.

[0069] Step (3), mapping band difference information to color space: mapping band difference information to the hue and saturation components of color space through color curves in color space, so that the band difference information is proportional to the visual difference of human eyes;

[0070] Step (4), adaptive dual histogram equalization of saturation mapping: calculate the mean value of saturation after mapping, and use the mean value as the dividing line to calculate the histogram of the part of pixel gray value less than the mean value and the histogram of the part of pixel gray value greater than the mean value respectively. The part less than the mean value is kept unchanged, and then the part greater than the mean value is subjected to histogram equalization.

[0071] Step (5), generation and conversion of LAB space color fusion image: the black and white fusion image generated in step (1) is used as the L component of LAB space color fusion image; then, based on the hue information in step (3) and the saturation information after equalization in step (4), the red-green component A and blue-yellow component B of LAB space color fusion image are calculated; finally, the color image in LAB space is converted to RGB space to obtain RGB space color fusion image.

[0072] In step (1), grayscale fusion can be performed using mean fusion or other suitable fusion algorithms.

[0073] In step (2), the difference in grayscale values ​​is used as the band difference information, and the difference information of band I and band II is extracted respectively. The specific method is as follows:

[0074] t(x,y)=LW(x,y)-MW(x,y),if LW(x,y)≥MW(x,y)

[0075] t(x,y)=MW(x,y)-LW(x,y),if MW(x,y)>LW(x,y)

[0076] In the formula, MW(x,y) and LW(x,y) are grayscale images of band I and band II, respectively, t(x,y) is the band difference information, and x and y represent the pixel position coordinates.

[0077] The specific method for step (3) is as follows:

[0078] First, perform tone mapping:

[0079] θ(x,y)=θ0, if MW(x,y)>LW(x,y)

[0080] θ(x,y)=θ0+π,if LW(x,y)≥MW(x,y)

[0081] In the formula, θ0 is the initial hue, which can be selected from 0 to 2π; θ(x,y) is the hue mapped to the color space;

[0082] Then, saturation mapping is performed, using the following formula:

[0083] r(x,y)=ct(x,y),if LW(x,y)≥MW(x,y)

[0084] r(x,y)=cy(x,y),if MW(x,y)>LW(x,y)

[0085] In the formula, r(x,y) is the saturation mapped to the color space, and c is the saturation mapping coefficient.

[0086] The specific method for step (4) is as follows:

[0087] First, calculate the mean and histogram of saturation:

[0088]

[0089] H(i) = n i

[0090] In the formula, mean_r(x,y) represents the mean saturation, M and N represent the total number of pixels in the x and y directions, respectively, i is the grayscale value, ranging from 0 to 255, H(i) is the histogram of saturation, and n i This represents the total number of pixels with a grayscale value of i. The histogram is divided into two parts: those below the mean and those above the mean, using the average value as the dividing line.

[0091] H_low(i l )=H(i≤mean_r)

[0092] H_high(i h )=H(i>mean_r)

[0093] In the formula, H_low(i l H_high(i) is a histogram of the portion of grayscale below the mean saturation. h (i) is a histogram of the portion of grayscale values ​​higher than the mean. l Gray values ​​below the mean, ranging from 0 to mean_r, i h Gray values ​​above the mean range from (mean_r+1) to 255. Values ​​below the mean are not processed. Then, the cumulative histogram of the values ​​above the mean is calculated, and histogram equalization is performed.

[0094]

[0095]

[0096] if r(x,y)=i hr_out(x,y)=HE(i h )

[0097] In the formula, CDF_high(i h HE(i) is the cumulative histogram of the portion above the mean. h (i) represents the histogram-equilibrated i h The corresponding new grayscale value, 'a' is the saturation control factor used to limit the maximum saturation value after equalization, and 'r_out(x,y)' is the final saturation image.

[0098] The specific method for step (5) is as follows:

[0099] The grayscale fusion image generated in step (1) is used as the L component of the LAB space color fusion image; then, the hue θ(x,y) generated in step (3) and the equalized saturation r_out(x,y) generated in step (4) are mapped to the A and B components of the LAB space color fusion image, as follows:

[0100] L(x,y)=K1×GF(x,y)

[0101] A(x,y)=K2×r_out(x,y)cos(θ(x,y))

[0102] B(x,y)=K3×r_out(x,y)sin(θ(x,y))

[0103] In the formula, L(x,y) is the luminance component L of the LAB space color fusion image, GF(x,y) is the grayscale fusion image obtained in step (1), A(x,y) is the red-green component A of the LAB space color fusion image, B(x,y) is the blue-yellow component B of the LAB space color fusion image, and K1, K2, and K3 are color space mapping coefficients. Finally, the LAB space color image is converted to RGB space to obtain the R, G, and B components of the color fusion image, which are used for image transmission and display.

[0104] Example 3

[0105] like Figure 1 As shown, a color image fusion algorithm that enhances band differences by utilizing the differences in human color vision includes the following steps:

[0106] Step (1), extraction and fusion of dual-band spatial information;

[0107] Step (2), band information difference extraction;

[0108] Step (3): Band difference information is mapped to color space;

[0109] Step (4): Map saturation adaptive dual histogram equalization;

[0110] Step (5): Generation and conversion of LAB space color fusion image.

[0111] Furthermore, step (1) specifically includes:

[0112] Spatial information from two infrared bands is extracted and grayscale fusion is performed. Grayscale fusion can be achieved using mean fusion or other suitable fusion algorithms. The main purpose of grayscale fusion is to preserve the high-frequency spatial information of the two bands. Methods such as multi-scale decomposition, sparse representation, and spatial domain fusion can be used for grayscale fusion.

[0113] Furthermore, step (2) specifically includes:

[0114] Using the difference in grayscale values ​​as band difference information, the difference information of band I and band II is extracted separately. The specific method is as follows:

[0115] t(x,y)=LW(x,y)-MW(x,y),if LW(x,y)≥MW(x,y)

[0116] t(x,y)=MW(x,y)-LW(x,y),if MW(x,y)>LW(x,y)

[0117] In the formula, MW(x,y) and LW(x,y) are grayscale images of band I and band II, respectively, t(x,y) is the band difference information, and x and y represent the pixel position coordinates.

[0118] Furthermore, step (3) specifically includes:

[0119] The band difference information is mapped to the hue and saturation components of the color space through color curves, making the band difference information proportional to the visual difference of the human eye; first, hue mapping is performed:

[0120] θ(x,y)=θ0, if MW(x,y)>LW(x,y)

[0121] θ(x,y)=θ0+π,if LW(x,y)≥MW(x,y)

[0122] In the formula, θ0 is the initial hue, which can be selected from 0 to 2π; θ(x,y) is the hue mapped to the color space;

[0123] Then, the difference information of band I and band II is proportionally mapped to the saturation of the corresponding hue:

[0124] r(x,y)=ct(x,y),if LW(x,y)≥MW(x,y)

[0125] r(x,y)=ct(x,y),if MW(x,y)>LW(x,y)

[0126] In the formula, r(x,y) is the saturation mapped to the color space, and c is the saturation mapping coefficient.

[0127] Furthermore, step (4) specifically includes:

[0128] First, calculate the mean and histogram of saturation:

[0129]

[0130] H(i) = n i

[0131] In the formula, mean_r(x,y) represents the mean saturation, M and N represent the total number of pixels in the x and y directions, respectively, i is the grayscale value, ranging from 0 to 255, H(i) is the histogram of saturation, and n i This represents the total number of pixels with a grayscale value of i. The histogram is divided into two parts: those below the mean and those above the mean, using the average value as the dividing line.

[0132] H_low(i l )=H(i≤mean_r)

[0133] H_high(i h )=H(i>mean_r)

[0134] In the formula, H_low(i l H_high(i) is a histogram of the portion of grayscale below the mean saturation. h (i) is a histogram of the portion of grayscale values ​​higher than the mean. l Gray values ​​below the mean, ranging from 0 to mean_r, i h Gray values ​​above the mean range from (mean_r+1) to 255. Values ​​below the mean remain unchanged. Then, the cumulative histogram of the values ​​above the mean is calculated, and histogram equalization is performed.

[0135]

[0136] if r(x,y)=i h r_out(x,y)=HW(i h )

[0137] In the formula, CDF_high(i h HE(i) is the cumulative histogram of the portion above the mean.h (i) represents the histogram-equilibrated i h The corresponding new grayscale value, 'a' is the saturation control factor used to limit the maximum saturation value after equalization, and 'r_out(x,y)' is the final saturation image.

[0138] Furthermore, step (5) specifically includes:

[0139] The LAB color space, based on the human eye's color vision, divides the color space into L (luminance) components, A (red / green) components, and B (blue / yellow) components. These three components have low correlation, which is beneficial for color control. When using this space for color fusion, images with L, A, and B components need to be generated based on the band characteristics of the dual-band image.

[0140] First, since the human eye is more sensitive to high-frequency signals in the luminance component, the black-and-white fusion image with high-frequency spatial information generated in step (1) is directly used as the L component of the color fusion image:

[0141] L(x,y)=K1×GF(x,y)

[0142] In the formula, L(x,y) is the luminance component L of the LAB space color fusion image, and GF(x,y) is the grayscale fusion image obtained in step (1).

[0143] Then, the hue θ(x,y) generated in step (3) and the equalized saturation r_out(x,y) generated in step (4) are mapped to the A and B components of the LAB space color fusion image, as follows:

[0144] A(x,y)=K2×r_out(x,y)cos(θ(x,y))

[0145] B(x,y)=K3×r_out(x,y)sin(θ(x,y))

[0146] In the formula, L(x,y) is the luminance component L of the LAB space color fusion image, GF(x,y) is the grayscale fusion image obtained in step (1), A(x,y) is the red-green component A of the LAB space color fusion image, B(x,y) is the blue-yellow component B of the LAB space color fusion image, and K1, K2, and K3 are color space mapping coefficients. Finally, the LAB space color image is converted to RGB space to obtain the R, G, and B components of the color fusion image, which are used for image transmission and display.

[0147] Example 4

[0148] This invention provides a color image fusion algorithm that utilizes differences in human color vision to enhance band differences. The processing flow is as follows: Figure 1As shown, the algorithm of this invention is applicable to the fusion of any two different dual-band images. As a demonstration, the following are the specific implementation steps using medium wave and long wave as examples:

[0149] Step 1, Dual-band spatial information extraction and fusion:

[0150] use Figure 2 The infrared dual-band image shown is used to extract spatial information from the two bands and then perform grayscale fusion. Grayscale fusion can employ simple mean fusion or other methods that better extract and preserve high-frequency spatial information from both bands, such as multi-scale decomposition, sparse representation, and spatial domain fusion. For simple verification, mean fusion is used here to achieve grayscale fusion. The calculation formula is as follows:

[0151]

[0152] In the formula, GF(x,y) is the grayscale fused image, and LW(x,y) and MW(x,y) are the long-wavelength grayscale image and the medium-wavelength grayscale image, respectively.

[0153] Step 2, band information difference extraction:

[0154] Compare the grayscale values ​​of the two bands, and use the difference in grayscale values ​​as the band difference information. Extract the long-wave difference information and the long-wave difference information separately. The specific method is as follows:

[0155] t(x,y)=LW(x,y)-MW(x,y),if LW(x,y)≥MW(x,y)

[0156] t(x,y)=MW(x,y)-LW(x,y),if MW(x,y)>LW(x,y)

[0157] In the formula, t(x,y) represents the band difference information, and x and y represent the pixel position coordinates.

[0158] Step 3: Mapping band difference information to color space:

[0159] The band difference information is mapped to the hue and saturation components of the color space through color curves, making the band difference information proportional to the visual difference of the human eye; first, hue mapping is performed:

[0160] θ(x,y)=θ0, if MW(x,y)>LW(x,y)

[0161] θ(x,y)=θ0+π,if LW(x,y)≥MW(x,y)

[0162] In the formula, θ0 is the initial hue, which can be selected from 0 to 2π, and is set to 0.3π here; θ(x,y) is the hue mapped to the color space;

[0163] Then, the band difference information is mapped proportionally to the saturation of the corresponding hue:

[0164] r(x,y)=ct(x,y),if LW(x,y)≥MW(x,y)

[0165] r(x,y)=ct(x,y),if MW(x,y)>LW(x,y)

[0166] In the formula, r(x,y) is the saturation mapped to the color space, and c is the saturation mapping coefficient, which is set to 1 here.

[0167] Step 4, Mapping Saturation Adaptive Bi-Histogram Equalization:

[0168] Directly mapped color saturation lacks scene adaptability, thus requiring saturation adjustment. However, directly using traditional image enhancement algorithms like histogram averaging can easily lead to overly vibrant background colors, resulting in a loss of spatial detail in the target. Therefore, using the mean saturation as a dividing line, the image histogram is divided into a background portion below the mean and a target portion above the mean. Histogram equalization is only performed on the portion above the mean. The specific method is as follows:

[0169] First, calculate the mean and histogram of saturation:

[0170]

[0171] H(i) = n i

[0172] In the formula, mean_r(x,y) represents the mean saturation, M and N represent the total number of pixels in the x and y directions, respectively, i is the grayscale value, ranging from 0 to 255, H(i) is the histogram of saturation, and n i This represents the total number of pixels with a grayscale value of i. The histogram is divided into two parts: those below the mean and those above the mean, using the average value as the dividing line.

[0173] H_low(i l )=H(i≤mean_r)

[0174] H_high(i h )=H(i>mean_r)

[0175] In the formula, H_low(i l H_high(i) is a histogram of the portion of grayscale below the mean saturation. h(i) is a histogram of the portion of grayscale values ​​higher than the mean. l Gray values ​​below the mean, ranging from 0 to mean_r, i h Gray values ​​above the mean range from (mean_r+1) to 255. Values ​​below the mean are not processed. Then, the cumulative histogram of the values ​​above the mean is calculated, and histogram equalization is performed.

[0176]

[0177] if r(x,y)=i h r_out(x,y)=HE(i h )

[0178] In the formula, CDF_high(i h HE(i) is the cumulative histogram of the portion above the mean. h (i) represents the histogram-equilibrated i h The corresponding new grayscale value, 'a' is the saturation control factor used to limit the maximum saturation value after equalization, which is set to 96 here, and 'r_out(x,y)' is the final saturation image.

[0179] Step 5, Generation and conversion of LAB space color fusion image:

[0180] The LAB color space, based on the human eye's color vision, divides the color space into L (luminance) components, A (red / green) components, and B (blue / yellow) components. These three components have low correlation, which is beneficial for color control. When using this space for color fusion, images with L, A, and B components need to be generated based on the band characteristics of the dual-band image.

[0181] First, since the human eye is more sensitive to high-frequency signals in the luminance component, the black-and-white fusion image with high-frequency spatial information generated in step (1) is directly used as the L component of the color fusion image:

[0182] L(x,y)=K1×GF(x,y)

[0183] In the formula, L(x,y) is the luminance component L of the LAB space color fusion image, and GF(x,y) is the grayscale fusion image obtained in step (1).

[0184] Then, the hue θ(x,y) generated in step (3) and the equalized saturation r_out(x,y) generated in step (4) are mapped to the A and B components of the LAB space color fusion image, as follows:

[0185] A(x,y)=K2×r_out(x,y)cos(θ(x,y))

[0186] B(x,y)=K3×r_out(x,y)sin(θ(x,y))

[0187] In the formula, L(x,y) is the luminance component L of the LAB space color fusion image, GF(x,y) is the grayscale fusion image obtained in step (1), A(x,y) is the red-green component A of the LAB space color fusion image, B(x,y) is the blue-yellow component B of the LAB space color fusion image, and K1, K2, and K3 are color space mapping coefficients, which are set to 0.39, 0.5, and 0.5 respectively here. Finally, the LAB space color image is converted to RGB space to obtain the R, G, and B components of the color fusion image, which are used for image transmission and display.

[0188] Mid-to-long band dual-band images were acquired using a dual-band infrared thermal imager, and simulation processing was performed using the aforementioned algorithm to obtain the following results: Figure 3 As shown in the results, this algorithm can achieve good color image fusion effects in different scenarios.

[0189] Most current dual-band color image fusion techniques operate within the RGB or HSV color spaces, while the technique proposed in this invention operates within the LAB color space. Compared to other commonly used color spaces, the LAB space offers better color accuracy, and a greater difference in LAB space distance can produce a stronger visual difference effect.

[0190] Meanwhile, traditional color image fusion techniques directly or indirectly transform and map dual-band information to the three channels of the color space through different methods. Although colorization is achieved, the relationship between color information and band information is unclear. Since the two bands often contain many similar or identical scene features, previous color fusion methods are prone to color casts, making it difficult to highlight the band differences of interest to the observer. The key to dual-band fusion technology lies in how to represent or enhance band difference information through fusion. This technology has thoroughly studied the mapping relationship between band differences and the color space. The proposed mapping method ensures that the human eye's color visual difference in the fused image is proportional to the corresponding band difference, guaranteeing the order preservation of the band difference mapping. For parts without difference, a colorless grayscale image is represented, ultimately achieving enhancement of band differences through human eye's color visual difference.

[0191] Furthermore, existing dual-band color image fusion techniques do not fully consider the significant variations in band differences across different application scenarios, only achieving good color image visual effects when the band differences within a scene are sufficiently large. This invention introduces a color adaptive adjustment mechanism into traditional color image fusion methods. It uses dual histograms to divide band differences into background and target, and then performs histogram equalization on the target's histogram, further enhancing the visual difference effect caused by the target's band differences in color image fusion. Ultimately, this invention, while preserving the mapping order of band difference information, fully utilizes the differences in human color vision to enhance band differences, achieving a color fusion effect with better visual perception and wider scene adaptability.

[0192] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A color image fusion algorithm that utilizes differences in human color vision to enhance band differences, characterized in that, Includes the following steps: Step (1), dual-band spatial information extraction and fusion: extract the spatial information of the two band infrared images and perform grayscale fusion; Step (2), band information difference extraction: compare the gray values ​​of the two bands, and use the difference in gray values ​​as the band difference information to extract the difference information of band I and band II respectively. Step (3), mapping band difference information to color space: mapping band difference information to the hue and saturation components of color space through color curves in color space, so that the band difference information is proportional to the visual difference of human eyes; Step (4), adaptive dual histogram equalization of saturation mapping: calculate the mean value of saturation after mapping, and use the mean value as the dividing line to calculate the histogram of the part of pixel gray value less than the mean value and the histogram of the part of pixel gray value greater than the mean value respectively. The part less than the mean value is kept unchanged, and then the part greater than the mean value is subjected to histogram equalization. Step (5), generation and conversion of LAB space color fusion image: the black and white fusion image generated in step (1) is used as the L component of LAB space color fusion image; then, based on the hue information in step (3) and the saturation information after equalization in step (4), the red-green component A and blue-yellow component B of LAB space color fusion image are calculated; finally, the color image in LAB space is converted to RGB space to obtain RGB space color fusion image.

2. The color image fusion algorithm for enhancing band differences by utilizing differences in human color vision, as described in claim 1, is characterized in that... In step (1), the grayscale fusion uses the mean fusion algorithm.

3. The color image fusion algorithm for enhancing band differences by utilizing differences in human color vision, as described in claim 1, is characterized in that... In step (2), for any two images with different bands, the difference in grayscale values ​​is used as the band difference information, and the difference information of band I and band II is extracted respectively. The specific method is as follows: t(x,y)=LW(x,y)-MW(x,y),if LW(x,y)≥MW(x,y) t(x,y)=MW(x,y)-LW(x,y),if MW(x,y)>LW(x,y) In the formula, MW(x,y) and LW(x,y) are grayscale images of band I and band II, respectively, t(x,y) is the band difference information, and x and y represent the pixel position coordinates.

4. The color image fusion algorithm for enhancing band differences by utilizing differences in human color vision, as described in claim 3, is characterized in that... The specific method for step (3) is as follows: First, perform tone mapping: θ(x,y)=θ0, if MW(x,y)>LW(x,y) θ(x,y)=θ0+π,if LW(x,y)≥MW(x,y) In the formula, θ0 is the initial hue, which can be selected from 0 to 2π; θ(x,y) is the hue mapped to the color space; Then, saturation mapping is performed, using the following formula: r(x,y)=ct(x,y),if LW(x,y)≥MW(x,y) r(x,y)=cy(x,y),if MW(x,y)>LW(x,y) In the formula, r(x,y) is the saturation mapped to the color space, and c is the saturation mapping coefficient.

5. The color image fusion algorithm for enhancing band differences by utilizing differences in human color vision, as described in claim 4, is characterized in that... The specific method for step (4) is as follows: First, calculate the mean and histogram of saturation: H(i)=n i In the formula, mean_r represents the mean saturation, M and N represent the total number of pixels in the x and y directions, respectively, i is the grayscale value, H(i) is the histogram of saturation, and n i This represents the total number of pixels with a grayscale value of i. Using the mean as the dividing line, the histogram is divided into two parts: below the mean and above the mean. H_low(i l )=H(i≤mean_r) H_high(i h )=H(i>mean_r) In the formula, H_low(i l H_high(i) is a histogram of the portion of grayscale below the mean saturation. h (i) is a histogram of the portion of grayscale values ​​higher than the mean. l Gray values ​​below the mean, ranging from 0 to mean_r, i h Gray levels above the mean are processed, while those below the mean are left unprocessed. Then, the cumulative histogram of the grayscale values ​​above the mean is calculated, and histogram equalization is performed. if r(x,y)=i h ,r_out(x,y)=HE(i h ) In the formula, CDF_high(i h HE(i) is the cumulative histogram of the portion above the mean. h (i) represents the histogram-equilibrated i h The corresponding new grayscale value, 'a' is the saturation control factor used to limit the maximum saturation value after equalization, and 'r_out(x,y)' is the final saturation image.

6. The color image fusion algorithm for enhancing band differences by utilizing differences in human color vision, as described in claim 5, is characterized in that... The specific method for step (5) is as follows: The grayscale fusion image generated in step (1) is used as the L component of the LAB space color fusion image; then, the hue θ(x,y) generated in step (3) and the equalized saturation r_out(x,y) generated in step (4) are mapped to the A and B components of the LAB space color fusion image; finally, the color image in LAB space is converted to RGB space to obtain the R, G, and B components of the color fusion image, which are used for image transmission and display.

7. The color image fusion algorithm for enhancing band differences by utilizing differences in human color vision, as described in claim 6, is characterized in that... The mapping algorithm between component A and component B is as follows: L(x,y)=K1×GF(x,y) A(x,y)=K2×r_out(x,y)cos(θ(x,y)) B(x,y)=K3×r_out(x,y)sin(θ(x,y)) In the formula, L(x,y) is the luminance component L of the LAB space color fusion image, GF(x,y) is the grayscale fusion image obtained in step (1), A(x,y) is the red-green component A of the LAB space color fusion image, B(x,y) is the blue-yellow component B of the LAB space color fusion image, and K1, K2 and K3 are color space mapping coefficients.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when executed by a processor, is capable of implementing the steps of a color image fusion algorithm as described in any one of claims 1-7, which utilizes differences in human eye color vision to enhance band differences.

Images