Identification and segmentation of high luminance information in night vision glare image

Abstract

This invention relates to a method for identifying and segmenting high-brightness information in night vision halo images. It includes the following steps: registering visible light and infrared images of a nighttime halo scene acquired simultaneously; and calculating adaptive iterative thresholds T for the brightness component Y of the registered visible light image and the infrared image, respectively. Y T I Extracting high-brightness information V from visible light and infrared images HB I HB The design of a heterogeneous semantic segmentation model acquires high-brightness halo information, useful high-brightness information, and useful high-brightness information from visible light and infrared images. Based on a fingerprint information matching mask association algorithm, it integrates missed identification information while eliminating misidentified information to obtain a complete and accurate high-brightness halo region α. H High-brightness useful information α L and low-light area α B This invention solves the problem that existing heterogeneous image fusion anti-halo methods fail to distinguish between high-brightness halos and high-brightness useful information, leading to the erroneous elimination of high-brightness useful information, and can eliminate potential safety risks for nighttime driving.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of night vision anti-halation technology, and mainly relates to a night vision halation image high-brightness information identification and segmentation method. BACKGROUND

[0002] The total traffic accident rate at night is 1.5 times that of the day due to poor lighting conditions, blocked driver's line of sight and other factors, and the traffic accidents caused by the dazzling of drivers due to the opening of high beams of oncoming vehicles account for 30%-40%. Under the night halation scene, the high-brightness information affecting driving safety not only includes the halation of vehicle headlights and the strong glare of road reflection, but also includes important traffic sign information such as lane lines and zebra crossings.

[0003] The document "Improved IHS-Curvelet Transform Fusion Anti-Halation Method of Visible Light and Infrared Image" uses the anisotropic support interval of Curvelet transform and designs a nonlinear adjustment strategy for the weight of infrared low-frequency coefficients to eliminate high-brightness halation. The document "Night Vision Anti-Halation Algorithm Based on Different-Source Image Fusion Combining Visual Saliency with YUV-FNSCT" uses the multi-direction, multi-scale and translation invariance of FNSCT decomposition to eliminate halation. However, the above two documents focus on eliminating halation by excluding high-brightness information to avoid its participation in fusion, and the halation elimination effect is good, but since the high-brightness halation and high-brightness useful information are not discriminated, the high-brightness halation and high-brightness useful information are involved in fusion according to the same rule, which leads to the elimination of high-brightness useful information while eliminating high-brightness halation, causing new potential safety hazards for night driving. SUMMARY

[0004] The purpose of the present application is to provide a night vision halation image high-brightness information identification and segmentation method, which overcomes the problem that the existing different-source image fusion algorithm does not distinguish between high-brightness halation and high-brightness useful information, resulting in the elimination of high-brightness useful information while eliminating high-brightness halation.

[0005] To achieve the above purpose, the technical scheme adopted by the present application is as follows: a night vision halation image high-brightness information identification and segmentation method, comprising the following steps:

[0006] Step 1, registering the visible light and infrared images of the night halation scene collected simultaneously to obtain a preprocessed image;

[0007] Step 2, performing YUV color space transformation on the registered visible light image to obtain three components of brightness Y, chroma U and saturation V; Step 3, performing a high-brightness halation elimination algorithm on the visible light image to obtain a visible light image without halation;

[0008] Step 3, respectively, for the luminance component Y, the registered infrared image is calculated adaptive iterative threshold T Y , T I , extract the high brightness information V HB , I HB ;

[0009] Step 4, design of heterogeneous semantic segmentation model, identify V HB , I HB , get the high brightness halo information of visible light image high brightness useful information and infrared high brightness useful information

[0010] Step 5, based on design mask correlation algorithm of fingerprint information matching, integrate the missed information and eliminate the misidentified information, get high brightness halo area α H , high brightness useful information area α L and low illumination area α B .

[0011] Further, in the above step 4, the design of the heterogeneous semantic segmentation model, specifically includes the following steps:

[0012] Step 4.1, the input of the registered visible light and infrared image is down-sampled by using the stack operation of convolution and pooling, and the receptive field is constantly increased while the convolution feature map U of the source image is extracted;

[0013] Step 4.2, the size of the convolution feature map U is expanded by using the bilinear interpolation method, and the up-sampling feature map V is obtained;

[0014] Step 4.3, the channel attention module is used to extract the shallow position information of the target;

[0015] Step 4.4, the spatial attention module is used to extract the deep semantic information of the target, and a new convolution feature map U' is obtained;

[0016] Step 4.5, the new convolution feature map U' and the corresponding up-sampling feature map V are spliced, and the up-sampling and feature fusion are repeated several times to complete the segmentation of high brightness pixels in visible light and infrared images, and the visible light halo area visible light high brightness useful information area and infrared high brightness information area

[0017] Further, the above step 5 includes the following specific steps:

[0018] Step 5.1, taking the ground truth of the source visible light and infrared image as the standard fingerprint information respectively And using a straight line segment as a basic marking processing unit, the connected domain is marked in a run mode;

[0019] Step 5.2, taking the visible light high-brightness useful information Infrared high-brightness useful information As the contrast fingerprint information And taking the connected domain as a unit, the matching degree D of the standard fingerprint and the contrast fingerprint is calculated H ;

[0020] Step 5.3, obtaining the to-be-associated information according to the relationship between the information matching degree D H And the matching threshold τ: when the matching degree D H Is greater than the threshold τ, it indicates that the current connected domain in the contrast fingerprint is a misrecognized area, so the current area is excluded and is not associated subsequently; otherwise, the current connected domain is reserved as the to-be-associated area;

[0021] Through the fingerprint information matching, the to-be-associated high-brightness information F i VI (x,y) of the i-th connected domain of the visible light and infrared image is obtained i IR (x,y) is obtained, and all the connected domains are traversed repeatedly to obtain the to-be-associated high-brightness useful information F VI (x,y) of the visible light and infrared image IR (x,y);

[0022]

[0023]

[0024] Step 5.4, integrating the to-be-associated information F VI (x,y) is obtained IR (x,y), the high-brightness halo area α H , the high-brightness useful information area α L And the low-illumination area α B

[0025] α L (x,y) = F VI (x,y) || F IR (x,y) (17)

[0026]

[0027]

[0028] Compared with the prior art, the present application has the advantages and effects that:

[0029] 1. The method of the present application effectively improves the accuracy and integrity of high-brightness information recognition in different night vision halo scenes by synchronously identifying the high-brightness halo and the high-brightness useful information of the heterogeneous images and correlating the high-brightness useful information of the two images, realizes the accurate division of the high-brightness halo area, the high-brightness useful information area and the low-illumination area, and effectively solves the problem of false elimination of high-brightness useful information existing in the existing anti-halo method of heterogeneous image fusion.

[0030] 2. The present application uses the characteristic that the pixel value of high-brightness information is much larger than that of low-illumination background area, first uses the adaptive threshold iteration high-brightness information extraction algorithm based on the pixel level to distinguish high-brightness information and low-illumination information, effectively avoids the influence of background noise, then uses the designed heterogeneous semantic segmentation model to realize pixel-by-pixel classification, adaptively extracts and connects the key point information of different channel characteristics, improves the segmentation accuracy of high-brightness halo and high-brightness useful information, and finally uses the mask correlation algorithm of fingerprint information matching to correlate the complementary segmentation results of high-brightness information of heterogeneous images using image fingerprint information, and improves the pixel integrity of high-brightness useful information. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a visible light image of a night halo scene;

[0032] Figure 2 is an infrared image of a night halo scene;

[0033] Figure 3 is a fusion image obtained by improving the IHS-Curvelet transform method;

[0034] Figure 4 is a fusion image obtained by combining YUV-FNSCT transform with visual saliency;

[0035] Figure 5 is a visible light image of a night halo scene with high-brightness information overlap;

[0036] Figure 6 is an infrared image of a night halo scene with high-brightness information overlap;

[0037] Figure 7 is the recognition result of high-brightness information by UNet semantic segmentation network;

[0038] Figure 8 is the recognition result of high-brightness information by UNet network combined with attention mechanism;

[0039] Figure 9 is the recognition result of high-brightness information by the present application;

[0040] Figure 10 is a workflow diagram of the present application. DETAILED DESCRIPTION

[0041] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0042] Referring to Figure 1 It can be seen that there are high-luminance glare and road surface reflection in the visible light image under the night vision glare scene, and the local high-luminance useful information (lane line) is hidden by the high-luminance reflection light, resulting in incomplete useful information. Referring to Figure 2 It can be seen that there is hidden high-luminance useful information in the infrared image under the night vision glare scene, that is, the high-luminance information of the heterogenous image has complementarity. Referring to Figure 3 and Figure 4 It can be seen that the existing heterogenous image fusion technology eliminates the high-luminance glare to some extent, but at the same time, the high-luminance useful information is weakened or mis-eliminated (compared with Figure 1 ), which brings new safety hazards for night driving.

[0043] The design idea of the present application is to identify the high-luminance information of the night glare scene pixel by pixel, segment the high-luminance glare, high-luminance useful information and low-illumination information, and use the information complementarity of the visible light and infrared images to correlate the identified high-luminance useful information, integrate the missed information and eliminate the mis-identified information, so as to obtain complete and accurate high-luminance glare, high-luminance useful information and low-illumination information.

[0044] The present application provides a method for identifying and segmenting high-luminance information of night vision glare image, comprising the following steps: registering the visible light image and infrared image of the night glare scene; performing YUV color space transformation on the registered visible light image to obtain three components of luminance Y, chrominance U and saturation V; extracting the high-luminance information of the luminance component Y and infrared image in a self-adaptive threshold iteration manner; inputting the extracted high-luminance information into a heterogenous semantic segmentation network for synchronous identification to obtain the high-luminance glare and high-luminance useful information of the visible light image, and the high-luminance useful information of the infrared image; finally, correlating the high-luminance useful information of the visible-infrared image according to the principle of fingerprint matching, integrating the missed information and eliminating the mis-identified information, to obtain complete and accurate high-luminance glare, high-luminance useful information and low-illumination information.

[0045] Embodiment

[0046] Referring to Figure 10The application provides a night vision halo image high-brightness information identification and segmentation method, which specifically comprises the following steps.

[0047] Step 1: registering the visible light image and the infrared image of the night halo scene synchronously collected to obtain a pretreatment image with high time and space consistency, and the specific steps of the embodiment are as follows:

[0048] (1) reading the visible light image and the infrared image of the night halo scene simultaneously by using an imread function;

[0049] (2) selecting four pairs of feature points from the two images by using a cpselect function and saving the feature points in base_points and input_points;

[0050] (4) obtaining a transformation matrix by using the obtained feature points and a used transformation type by using a cp2tform function;

[0051] (5) applying the transformation matrix to the image to be registered to perform affine transformation by using an imtransform function;

[0052] (6) cutting the visible light image to the same size as the infrared image by using an imcrop function.

[0053] Step 2: performing YUV color space transformation on the registered visible light image VI to obtain three components of brightness Y, chroma U and saturation V. The conversion formula from the RGB model to the YUV model is as follows:

[0054]

[0055] Step 3: obtaining adaptive iterative thresholds T Y and T I of the brightness component Y and the infrared image IR respectively, and extracting high-brightness information V HB and I HB of the visible light image and the infrared image.

[0056] Step 3.1: the adaptive iterative threshold T i+1 of the i+1th time is as follows:

[0057] T i+1 = m (μ1+μ2) (2)

[0058] In the formula, m is an adaptive coefficient, μ1 and μ2 are pixel mean values of the high-brightness pixel region and the background region segmented by the threshold T i , and are expressed as:

[0059]

[0060] In the formula, In(j) is the gray value of the jth pixel, N is the total number of pixels, N1 and N2 are the number of pixels of the two regions respectively, and N=N1+N2.

[0061] The initial threshold value T0 takes the median value of the pixels of the image to iteratively calculate until the threshold value no longer changes, and the latest threshold value is the critical pixel threshold value T.

[0062] Step 3.2, extracting the high brightness information a of the image according to the critical pixel threshold value T HB :

[0063]

[0064] Step 4, designing a heterogeneous semantic segmentation model to identify the high brightness information V of the visible light and infrared images HB , HB , obtaining the high brightness halo information of the visible light image high brightness useful information and infrared high brightness useful information

[0065] The heterogeneous semantic segmentation model uses an encoding-decoding structure to complete feature extraction and morphological segmentation of high brightness pixels, specifically including the following steps:

[0066] Step 4.1, using a convolution plus pooling stacking operation to downsample the input visible light-infrared image, constantly increasing the receptive field while extracting the convolution feature map U of the source image. The convolution kernel size in the convolution layer is 3*3, the number of convolution kernels is 64, 128, 256 and 512, and the activation function Relu is represented as:

[0067] f(x)=max(0,x) (5)

[0068] Step 4.2, using a bilinear interpolation method to expand the size of the feature map U to obtain the upsampled feature map V.

[0069] Step 4.3, using a channel attention module to extract the shallow position information of the target.

[0070] The convolution feature map U (size HxWxC) is subjected to global maximum pooling and average pooling to obtain two 1x1xC feature maps, the results of which are input into two layers of weight-shared perceptron MLP, the output features are added and subjected to an activation function to generate channel attention weights M C . Finally, the input feature map is multiplied by the channel attention weights to generate a channel attention feature map F', represented as:

[0071] F'=M c ×U (6)

[0072] Mc = σ [f2(δ(f1(Z avg )))+f2(δ(f1(Z max )))] (7)

[0073] Z avg = AvgPool(U) (8)

[0074] Z max = MaxPool(U) (9)

[0075] In the formula, AvgPool represents global average pooling, which calculates the average value of pixels in each channel; MaxPool represents global maximum pooling, which retains the maximum value of pixels in each channel of the feature map; f1 is a fully connected layer with input channel C and output channel C / 16; f2 is a fully connected layer with input channel C / 16 and output channel C; δ is a ReLU function, and σ is a Sigmoid function, which is represented as:

[0076]

[0077] Step 4.4, extract the deep semantic information of the target by using the spatial attention module, and obtain a new convolution feature map U'.

[0078] Take the channel attention feature map F' as input, first perform average pooling and maximum pooling on the channel dimension to obtain two HxWx1 feature maps, then concatenate the two feature maps and pass them through a 7x7 convolution layer, and finally pass the result through a Sigmoid function to obtain a two-dimensional single-channel spatial attention weight M s .

[0079] Finally, multiply the spatial attention weight M s and the channel attention feature map F' to generate a feature map F" with both spatial and channel attention, so as to suppress invalid features and focus only on the target area.

[0080] The new convolution feature map U' is represented as:

[0081] U' = F" = M s x F' (11)

[0082] M s = σ [f3(Z AVG ,Z MAX ))] (12)

[0083] Z AVG = AVGPool(U) (13)

[0084] Z MAX = MAXPool(U) (14)

[0085] In the formula, AVGPool represents channel dimension average pooling, the corresponding pixel of each channel feature map is added and averaged; MAXPool represents channel dimension maximum pooling, the maximum pixel value of the corresponding position of each channel feature map is retained; f3 is a convolution with a size of 7*7, and the output channel is 1.

[0086] Step 4.5, splice the new convolution feature map U' with the corresponding up-sampling feature map V, and repeat the up-sampling and feature fusion 3-5 times to complete the segmentation of high-intensity pixels in the visible light and infrared images, and obtain the visible light halo area Visible light high-intensity information And infrared high-intensity information In this embodiment, it is 4 times.

[0087] Step 5, based on the obtained visible light, infrared complementary high-intensity useful information Design a mask correlation algorithm for matching fingerprint information, and integrate the missed information and eliminate the misrecognized information through mask correlation, and finally obtain complete and accurate high-intensity halo area a H , high-intensity useful information a L And low-illumination area a B . Specifically, the following steps are included:

[0088] Step 5.1, respectively, with the Ground Truth of the source visible light-infrared image as the standard fingerprint information And use the bwlabel function to mark the connected domain, which uses a straight line segment as the basic processing unit of the run method to mark, fully utilizes the structure information of the region connection, and reduces the neighborhood logical judgment.

[0089] Step 5.2, the segmentation mask of visible light, infrared high-intensity useful information As the control fingerprint information And take the connected domain as the unit, use the pdist function to return the Hamming distance between the corresponding connected domains of the standard and control fingerprints, that is, the information matching degree D H .

[0090] Suppose And Correspond to And The vector space of the i-th connected domain, then the information matching degree Is represented as the total number of different vector values.

[0091] Step 5.3, according to the relationship between the information matching degree D H And the matching threshold τ, the information to be associated is obtained. Set the matching threshold τ, if the matching degree D Hgreater than threshold τ, indicating that the current connected domain in the control fingerprint is a false recognition area, so the current area is eliminated and not associated subsequently; if the matching degree D H less than threshold τ, the current vector in the control fingerprint is taken as the area to be associated.

[0092] The information F i VI (x,y) to be associated of the i-th connected domain of the visible light and infrared images is obtained by matching the fingerprint information. i IR (x,y) are respectively expressed as:

[0093]

[0094]

[0095] wherein, are respectively the control fingerprints corresponding to the i-th connected domain of the visible light and infrared images. All connected domains are traversed repeatedly to obtain the high-brightness useful information F VI (x,y) to be associated of the visible light and infrared images. IR (x,y).

[0096] Step 5.4, integrate the information F VI (x,y) to be associated. IR (x,y) to obtain the complete and accurate high-brightness halo area α H , high-brightness useful information area α L and low-illumination area α B .

[0097] The high-brightness useful information area α L (x,y) is generated by integrating the complementary mask information with high matching degree in the heterogeneous images using logical OR operation.

[0098] α L (x,y) = F VI (x,y) || F IR (x,y) (17)

[0099] The high-brightness halo area α H and the low-illumination area α B are respectively expressed as:

[0100]

[0101]

[0102] wherein, Y(x,y) is the pixel value of the visible light luminance component Y at point (x,y).

[0103] To verify the effectiveness of this invention in accurately and completely identifying high-brightness information in night vision halo images, experiments were conducted comparing the high-brightness information identification methods of the UNet semantic segmentation network (Algorithm1), the UNet semantic segmentation network (Algorithm2) incorporating the CBAM convolutional attention mechanism, and the method of this invention on acquired visible light and infrared images of night vision halo scenes. Simulation conditions: Windows 10 operating system, MATLAB software, PyCharm software.

[0104] The experimental results were evaluated using both subjective and objective methods. Two objective metrics were selected: MAP and MIoU. MAP is the average pixel accuracy, which is the average ratio of the number of correctly classified pixels in each class to the total number of pixels in that class. MIoU is the average intersection-over-union ratio, which is the average intersection-over-union ratio of pixels in each class with the Ground Truth. They are expressed as follows:

[0105]

[0106]

[0107] This example uses a night vision halo scene with completely overlapping high-brightness information as an example, and its visible light image (see...) Figure 5 In the infrared image, the halo effect is quite severe, and some high-brightness useful information, such as lane lines, is obscured by the halo reflected from the road surface, resulting in a large area of ​​overlap between the two; while in the infrared image (see... Figure 6 Unaffected by high-brightness halos, and the useful high-brightness information (lane lines) has a clear outline. The results of high-brightness information recognition and segmentation obtained by Algorithm 1, Algorithm 2, and the method of this invention (see...) Figure 7 , Figure 8 , Figure 9 The objective evaluation indicators are shown in Table 1.

[0108] Table 1 Objective evaluation indicators for night vision halo image recognition and segmentation results

[0109]

[0110] Depend on Figure 7 It can be seen that Algorithm 1 can only identify high-brightness halos, but not useful information about high brightness. (From...) Figure 8 It can be seen that Algorithm 2 can accurately identify unobstructed high-brightness halos and useful information, but it cannot obtain useful information that is obscured by road surface reflected light.

[0111] Depend on Figure 9 It can be seen that this invention not only accurately segments the unobstructed high-brightness halo and useful information, but also recovers the partially obscured useful information. (Compared to...) Figure 7Compared with the prior art, the segmentation of the high-brightness halo region is more accurate. Figure 8 Compared with the prior art, the segmentation result of the high-brightness useful information is more complete, which indicates that the accuracy and completeness of the high-brightness information recognition and segmentation can be ensured for the worse night vision halo scene.

[0112] As shown in Table 1, the MAP of the recognition result of Algorithm 2 is improved by 8.84% compared with Algorithm 1, and the MIoU is improved by 4.64% compared with Algorithm 1. The reason is that the attention module introduced by the feature fusion network can suppress invalid features, so that the network pays more attention to the target region features.

[0113] The MAP of the algorithm in the present application is improved by 12.41% and 3.57% compared with Algorithm 1 and Algorithm 2 respectively, and the MIoU is improved by 6.91% and 2.27% compared with Algorithm 1 and Algorithm 2 respectively. The reason is that the high-brightness information extraction used in the present application can avoid the interference of low-illumination information on the recognition process, so the MAP index is the highest; and the mask correlation algorithm of the fingerprint information matching designed in the present application can integrate the complementary high-brightness information of visible light-infrared, so the MIoU index is the highest.

[0114] From the above subjective and objective analysis, it can be seen that the recognition and segmentation method of the night vision halo image proposed in the present application can accurately divide the high-brightness halo, the high-brightness useful information and the low-illumination information. Applied to the anti-halo field of heterogeneous image fusion, different fusion rules can be set according to the fusion mechanism of different regions, so as to reasonably eliminate the high-brightness halo information while retaining the high-brightness useful information, avoid potential safety hazards, and improve the safety of night driving.

[0115] The above embodiments only exemplarily illustrate the principles and effects of the present application, and those skilled in the art can make some modifications and improvements without departing from the concept of the present application, which are all within the protection scope of the present application.

Claims

1. A method for identifying and segmenting high-brightness information of a night halo light image, comprising the following steps: Step 1, registering the visible light and infrared images of the night halo light scene collected simultaneously to obtain a preprocessed image; Step 2, performing YUV color space transformation on the registered visible light image to obtain three components of brightness Y, chroma U and saturation V; Step 3, adaptive iterative threshold T is calculated for the luminance component Y and the registered infrared image respectively Y , T I , high luminance information V HB 、I HB of the visible light and infrared images is extracted; Step 4, design a heterogeneous semantic segmentation model to identify V HB , HB Obtain high-brightness halo information of the visible light image Visible light high-brightness useful information φ VL And infrared high-brightness useful information Step 5, based on A mask correlation algorithm for fingerprint information matching is designed to integrate the missed information and eliminate the misrecognized information, so as to obtain a high-brightness halo area α H , a high-brightness useful information area α L , and a low-illumination area α B ; In step 4, the design of the heterogeneous semantic segmentation model specifically comprises the following steps: Step 4.1, performing a stack operation of convolution and pooling on the input registered visible light and infrared images to perform down-sampling, constantly increasing the receptive field while extracting the convolution feature map U of the source image; Step 4.2, expanding the size of the convolution feature map U by using a bilinear interpolation method to obtain an up-sampling feature map V; Step 4.3, extracting the shallow position information of the target by using a channel attention module; Step 4.4, extracting the deep semantic information of the target by using a spatial attention module to obtain a new convolution feature map U'; Step 4.5, the new convolution feature map U' is spliced with the corresponding up-sampling feature map V, and the up-sampling and feature fusion are repeated for multiple times to complete the segmentation of high-brightness pixels in the visible light and infrared images, and high-brightness halo information of the visible light image is acquired useful information of visible light high brightness and useful information of infrared high brightness The step 5 comprises the following specific steps: Step 5.1, using the ground truth of the source visible-infrared image as the standard fingerprint information, respectively and using a straight line segment as a basic marking unit to mark the connected domain in a run mode. Step 5.2, visible high-light useful information Infrared high-light useful information As a control fingerprint information And in the unit of connected domain, the matching degree D of the standard fingerprint and the control fingerprint is calculated H ; Step 5.3, the matching degree D of the reference fingerprint H The relationship with the matching threshold τ obtains the information to be associated: when the D of the reference fingerprint H is greater than the threshold τ, it indicates that the current connected domain in the reference fingerprint is a misrecognized area, so the current area is eliminated and is not associated subsequently; otherwise, the current connected domain is reserved as the area to be associated. Obtain the high-brightness information F to be associated of the i-th connected domain of the visible light and infrared image through the fingerprint information matching i VI (x, y), F i IR (x, y), and repeat the traversal of all connected domains to obtain the high-brightness useful information F to be associated of the visible light and infrared image VI (x, y), F IR (x, y) wherein, respectively, are the control fingerprints corresponding to the i-th connected component of the visible light and infrared images; denotes the total number of different vector values, and respectively correspond to and the vector space of the i-th connected component of the above. Step 5.4, integrate the obtained visible light, infrared image to be associated highlight useful information F VI (x,y), F IR (x,y), obtain highlight halo area α H , highlight useful information α L and low light area α B a L (x,y) = F VI (x,y) || F IR (x,y) Wherein, Y(x,y) is the pixel value of the visible light brightness component Y at point (x,y).

Images