An infrared dual-band and visible multi-feature transfer image fusion method

Abstract

The application discloses an infrared dual-waveband and visible light multi-feature transfer image fusion method to solve the problem that the infrared feature of the existing infrared and visible light fused image is relatively single, which results in low visual effect and target saliency, and weak target detection, identification and tracking performance. Meanwhile, a long-wave infrared gray image, a medium-wave infrared gray image and a visible light color image are processed, and the gradient, contrast and brightness features of the long-wave infrared are transferred to the visible light image. A temperature difference color image is constructed from the temperature difference information of the medium-wave infrared, and finally, the visible and infrared images are fused in regions. The method transfers multiple features of the infrared image, that is, the temperature difference information of the infrared image is displayed, and meanwhile, the texture information of the visible light image is reserved. Compared with the traditional method of only transferring and fusing the infrared brightness information, the method transfers multiple features of the infrared, so that the fused image is rich in information features, which is beneficial to observation, tracking and identification in the later stage.

Description

TECHNICAL FIELD

[0001] The present application relates to an image fusion method, in particular to a multi-feature transfer image fusion method of infrared dual-band and visible light. BACKGROUND

[0002] With the continuous development of digital image processing technology, infrared image and visible light image fusion technology as a key step in the image acquisition processing and display process, also in the continuous development and progress.

[0003] The commonly used infrared and visible light image fusion method is infrared and visible dual-band fusion. Usually, the brightness feature of the infrared image and the visible light background texture are selected to constitute a new fusion image. However, temperature information is the main feature of the infrared image, in addition to temperature information, there are other features, such as contrast, edge feature, etc. However, the existing infrared and visible light image fusion method cannot integrate these features into visible light, so that the existing infrared and visible light fusion image has relatively single infrared feature, resulting in low visual effect and target saliency, and the target detection, recognition and tracking performance are also weak. SUMMARY

[0004] The purpose of the present application is to provide a multi-feature transfer image fusion method of infrared dual-band and visible light, so as to solve the technical problems that the existing infrared and visible light fusion image has relatively single infrared feature, resulting in low visual effect and target saliency, and the target detection, recognition and tracking performance are also weak.

[0005] In order to achieve the above purpose, the present application provides a multi-feature transfer image fusion method of infrared dual-band and visible light, which is characterized by comprising the following steps:

[0006] Step 1, registering a long-wave infrared gray image I lwir , a medium-wave infrared gray image I mwir and a visible light color image I vis of the same aerial target scene;

[0007] Step 2, converting the visible light color image I vis from RGB space to HSV space, separating out the V channel to obtain a brightness component image V vis , and extracting a gradient image G lwir in the long-wave infrared gray image I lwir , superimposing the brightness component image V vis and the gradient image G lwir to generate a fused gradient brightness component image V gra ;

[0008] Step 3, according to the long-wave infrared gray image I lwirbackground gray value and target gray value of the long-wave infrared gray image I gra vis vis

[0009] Step 4, the long-wave infrared gray image I lwir is segmented into three regions of background region, target normal temperature region and target high temperature region by using different gray threshold values T1 and T2, and background region binary mask B1, target normal temperature region binary mask B2 and target high temperature region binary mask B3 are generated.

[0010] Step 5, a temperature difference color image I mwir is generated according to the temperature difference information of the medium-wave infrared gray image I infra .

[0011] Step 6, the background region binary mask B1, the target normal temperature region binary mask B2 and the target high temperature region binary mask B3 generated in step 4 are used to linearly superimpose the visible light color image I' vis migrated in step 3 and the temperature difference color image I infra generated in step 5 in different regions with different proportions to generate a final fusion image.

[0012] Further, step 3 specifically includes:

[0013] 3.1, obtaining the background gray value and the target gray value of the long-wave infrared gray image I lwir .

[0014] 3.2, stretching and translating the luminance component image V gra after gradient fusion in step 2 according to the following formula to generate a new luminance component image V' vis .

[0015]

[0016] In the formula, T bg is the target average gray value; V gra (i,j) is the gray value at the luminance component coordinate (i,j); k is the stretching coefficient, and m is the translation factor.

[0017] 3.3, converting the new luminance component image V' vis back to the RGB space to generate the visible light color image I' vis migrated.

[0018] Further, the region mask in step 4 is obtained according to the following formula:

[0019] ​​​

[0020] where (x, y) is the image coordinate.

[0021] Further, step 5 is specifically:

[0022] 5.1, convert the mid-wave infrared gray image I mwir to V brightness component;

[0023] 5.2, set different saturation values for different regions according to the difference of high-temperature and normal-temperature gray values, and select the same hue:

[0024]

[0025] where H is the hue; H1 is the selected hue; S is the saturation; S1, S2 and S3 represent different saturation values.

[0026] 5.3, convert HSV back to RGB space to generate the temperature difference color image I infra .

[0027] Further, step 6 is specifically:

[0028] 6.1, linearly superimpose the three channels of the visible light color image I' vis migrated in step 3 and the temperature difference color image I infra generated in step 5 in the RGB space respectively with different coefficients to obtain the images I F1 , I F2 and I F3 with different proportion coefficient fusion according to the following formula:

[0029]

[0030] 6.2, merge the fusion images of different regions to generate the final fusion image I F :

[0031] I F = B1.*I F1 + B2.*I F2 + B3.*I F3 .

[0032] The beneficial effects of the present application are:

[0033] 1. The infrared dual-band and visible light multi-feature migration image fusion method provided by the application simultaneously processes a long-wave infrared gray image, a medium-wave infrared gray image and a visible light color image, and migrates the gradient, contrast and brightness features of the long-wave infrared to the visible light image. A temperature difference color image is constructed from the temperature difference information of the medium-wave infrared, and finally the visible and infrared images are fused in regions. The method migrates multiple features of the infrared image, i.e. displays the temperature difference information of the infrared image, while retaining the texture information of the visible light image. Compared with the traditional method of only migrating and fusing the brightness information of the infrared, the method migrates multiple features of the infrared, so that the fused image has rich information features, which is beneficial to later observation, tracking and identification.

[0034] 2. Compared with the fused image of the prior art, the fused image of the application has enhanced edge, brightness, contrast and temperature difference. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a flowchart of an infrared dual-band and visible light multi-feature migration image fusion method of the application;

[0036] Figure 2 is a fused image diagram, wherein figure (a) adopts the infrared dual-band and visible light multi-feature migration image fusion method provided by the application, and figure (b) adopts the existing image fusion method. DETAILED DESCRIPTION

[0037] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments of the application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the application.

[0038] As shown in Figure 1 , an infrared dual-band and visible light multi-feature migration image fusion method comprises the following steps:

[0039] Step 1, registering a long-wave infrared gray image I lwir , a medium-wave infrared gray image I mwir and a visible light color image I vis of the same aerial target scene;

[0040] Step 2, converting the visible light color image I vis from an RGB space to an HSV space, separating out a V channel brightness component image V vis , and extracting a gradient image G lwir of the long-wave infrared gray image I lwir; Superimposed brightness component image V vis and gradient image G lwir Generate the brightness component image V after fusion gradient. gra Gradient image G lwir Calculate according to the formula:

[0041]

[0042] Among them, G x =I lwir (x+1,y)-I lwir (x+1,y); G y =I lwir (x,y+1)-I lwir (x,y)

[0043] Step 3: Combine the brightness component image V after gradient fusion. gra Based on long-wave infrared grayscale image I lwir V' is generated by grayscale stretching and translation of background and target grayscale values. vis Then convert it back to RGB space to generate the migrated visible light color image I' vis Specifically, this includes:

[0044] 3.1) Acquire long-wave infrared grayscale image I lwir The background grayscale value is 125 and the target grayscale value is 170;

[0045] 3.2) For the luminance component V gra Stretch and translate according to the formula to generate a new luminance component V'. vis ;

[0046]

[0047] Among them, T bg Choose 170, k = 12, m = 35;

[0048] 3.3) The new luminance component V' vis Convert back to RGB color space and obtain the migrated visible light color image I' vis .

[0049] Step 4: Using different grayscale thresholds T1 and T2, the long-wave infrared grayscale image I... lwir The target area is divided into three regions: a background region, a target room temperature region, and a target high temperature region. A binary mask B1 is generated for the background region, a binary mask B2 for the target room temperature region, and a binary mask B3 for the target high temperature region. The partition masks are obtained using the following formula:

[0050]

[0051] Where (x, y) is the image coordinate.

[0052] Step 5, generating a temperature difference color image I from the mid-wave infrared gray image temperature difference information infra Specifically,

[0053] 5.1), the mid-wave infrared gray image I mwir is taken as the V luminance component;

[0054] 5.2, different saturation values are set for different regions according to the difference between the high-temperature and normal-temperature gray values, and a hue is selected.

[0055]

[0056] Where H is the hue; H1 is the selected hue; S is the saturation; S1, S2, and S3 represent different saturation values. T1 is taken as 150, T2 is taken as 230, S1 = 230, S2 = 100, S3 = 50, and H1 = 0.03.

[0057] 5.3), the HSV is converted back to the RGB space to generate a temperature difference color image I infra .

[0058] Step 6, the migrated visible light color image I' vis and the temperature difference color image I infra are linearly superimposed in different proportions according to the background area, the target normal-temperature area, and the target high-temperature area in different regions to generate a final fusion image. Specifically,

[0059] 6.1), the RGB space three channels of the migrated color image I' vis and the temperature difference color image I infra are linearly superimposed in different proportions according to the formula to obtain the images I F1 , I F2 , and I F3 fused in different proportion coefficients respectively.

[0060]

[0061] The background area selects the migrated color image, the target normal-temperature area is superimposed according to the color image and the temperature difference color image in a proportion of 6 / 4, and the high-temperature area is superimposed according to the color image and the temperature difference color image in a proportion of 2 / 8.

[0062] 6.2), the fusion images of different regions are merged according to the formula to generate a final fusion image I F .

[0063] I F = B1.*I F1 + B2.*I F2 + B3.*IF3 .

[0064] As shown in Figure 2 Fig. (a) and Fig. (b) compared, the target image obtained by using the image fusion method described above, its image edge, brightness, contrast and temperature difference degree are significantly enhanced, so that its visual effect and target degree are greatly improved, on this basis, target detection recognition and tracking performance will be greatly enhanced.

[0065] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this, any change or replacement within the technical scope disclosed by the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A multi-feature transfer image fusion method of infrared dual-band and visible light, characterized in that, The method comprises the following steps: Step 1, registering a long-wave infrared gray-level image I of the same aerial target scene lwir , a mid-wave infrared gray-level image I mwir , and a visible light color image I vis ; Step 2, the visible light color image I vis is converted from RGB space to HSV space, the V channel is separated, and a luminance component image V is obtained vis , and a gradient image G in the long-wave infrared gray image I lwir is extracted lwir , the luminance component image V vis and the gradient image G lwir are superimposed to generate a fused gradient luminance component image V gra ; Step 3, according to the background gray value and target gray value of the long-wave infrared gray image I lwir , the gray stretching and translation are performed on the luminance component image V fused with the gradient in Step 2 to generate a new luminance component image V' gra , and then the image is converted back to the RGB space to generate the visible light color image I' vis after migration vis ; Step 4, dividing the long-wave infrared gray-scale image I lwir into three regions of a background region, a target normal-temperature region, and a target high-temperature region, and generating a background region binary mask B1, a target normal-temperature region binary mask B2, and a target high-temperature region binary mask B3; Step 5, generating a temperature difference color image I mwir from the temperature difference information of the mid-wave infrared grayscale image I infra ; Step 6, based on the background region binary mask B1 generated in step 4, the target normal temperature region binary mask B2 and the target high temperature region binary mask B3 generated in step 5, the visible color image I' after migration in step 3 is processed vis and the temperature difference color image I generated in step 5 infra Linearly superimposed in different regions with different proportions to generate the final fusion image.

2. The infrared dual-band and visible multi-feature transfer image fusion method according to claim 1, characterized in that, Step 3 specifically comprises: 3.1, obtaining a long-wave infrared gray-scale image I lwir background gray value and target gray value; 3.

2. The luminance component image V after fusing the gradient in step 2 gra Stretch and shift according to the following formula to generate a new luminance component image V' vis : where T bg is the target average gray level; V gra (i,j) is the gray level at luminance component coordinate (i,j); k is a stretching coefficient and m is a translation factor. 3.

3. The new luminance component image V' is computed as vis Back to the RGB space, the transferred visible color image I' is generated vis .

3. The multi-feature transfer image fusion method of infrared dual-band and visible light according to claim 1 or 2, characterized in that, In step 4, the partition mask is obtained according to the following formula: In the formula, (x, y) is an image coordinate.

4. The infrared dual-band and visible multi-feature transfer image fusion method according to claim 3, characterized in that, Step 5 specifically comprises: 5.1, the mid-wave infrared gray-scale image I mwir as the V luminance component; 5.2, different saturation values are set for different regions according to the difference between high-temperature and normal-temperature gray value, and a same hue is selected: In the formula, H is a hue; H1 is a selected hue; S is a saturation; S1, S2 and S3 represent different saturation values; 5.

3. Convert HSV back to RGB space to generate the temperature difference color image I infra .

5. The infrared dual-band and visible multi-feature transfer image fusion method according to claim 4, characterized in that, Step 6 specifically comprises: 6.1, visible color image I' after migration in step 3 vis and the temperature difference color image I generated in step 5 infra The three channels of the RGB space are respectively linearly superimposed with different coefficients to obtain the image I fused with different proportional coefficients F1 , F2 I F3 , and I 6.2, merging the fusion images of different regions to generate a final fusion image I F : I F = B1 * I F1 + B2 * I F2 + B3 * I F3 .

Images